论文标题
用于对嘈杂标签的神经网络进行强有力训练的核心
Coresets for Robust Training of Neural Networks against Noisy Labels
论文作者
论文摘要
现代神经网络具有过度贴合现实数据集中经常发现的嘈杂标签的能力。尽管取得了巨大进展，但现有技术在提供嘈杂标签训练的神经网络提供理论保证方面受到限制。在这里，我们提出了一种新颖的方法，并具有强大的理论保证，以对训练有嘈杂标签的深层网络进行强有力的培训。我们方法背后的关键思想是选择清洁数据点的加权子集（核心），这些子集（核心）提供了大约低级别的雅各布矩阵。然后，我们证明应用于子集的梯度下降不会使嘈杂的标签过于拟合。我们的广泛实验证实了我们的理论，并证明了在我们的子集进行训练的深网络中，与最先进的情况相比，在CIFAR-10上的准确性增加了6％，具有80％的噪声标签，而Mini Webvision的准确性提高了7％。
Modern neural networks have the capacity to overfit noisy labels frequently found in real-world datasets. Although great progress has been made, existing techniques are limited in providing theoretical guarantees for the performance of the neural networks trained with noisy labels. Here we propose a novel approach with strong theoretical guarantees for robust training of deep networks trained with noisy labels. The key idea behind our method is to select weighted subsets (coresets) of clean data points that provide an approximately low-rank Jacobian matrix. We then prove that gradient descent applied to the subsets do not overfit the noisy labels. Our extensive experiments corroborate our theory and demonstrate that deep networks trained on our subsets achieve a significantly superior performance compared to state-of-the art, e.g., 6% increase in accuracy on CIFAR-10 with 80% noisy labels, and 7% increase in accuracy on mini Webvision.
