论文标题
用于过度参数学习的实验设计，并应用于单镜头深度积极学习
Experimental Design for Overparameterized Learning with Application to Single Shot Deep Active Learning
论文作者
论文摘要
现代机器学习模型表现出的令人印象深刻的性能取决于能够在大量标记的数据上训练此类模型的能力。但是，由于获取大量标记的数据通常受到限制或昂贵，因此希望通过仔细策划训练集来减轻这种瓶颈。最佳实验设计是一个完善的范式，用于选择要标记的数据点，以最大程度地告知学习过程。不幸的是，有关最佳实验设计的经典理论着重于选择示例，以学习范式不足（因此是非相互关系）模型，而现代的机器学习模型（例如深度神经网络）被过度参数化，并且经常训练以插值训练。因此，经典的实验设计方法不适用于许多现代学习设置。实际上，缺乏参数化模型的预测性能往往是差异的，因此经典的实验设计侧重于降低方差，而过度参数化模型的预测性能也可以是本文所示，偏见是偏见或混合性质。在本文中，我们提出了一种设计策略，该策略非常适合过度参数化的回归和插值，我们通过提出一种用于单个Shot Deep Attive Active学习的新算法来证明我们方法在深度学习中的适用性。
The impressive performance exhibited by modern machine learning models hinges on the ability to train such models on a very large amounts of labeled data. However, since access to large volumes of labeled data is often limited or expensive, it is desirable to alleviate this bottleneck by carefully curating the training set. Optimal experimental design is a well-established paradigm for selecting data point to be labeled so to maximally inform the learning process. Unfortunately, classical theory on optimal experimental design focuses on selecting examples in order to learn underparameterized (and thus, non-interpolative) models, while modern machine learning models such as deep neural networks are overparameterized, and oftentimes are trained to be interpolative. As such, classical experimental design methods are not applicable in many modern learning setups. Indeed, the predictive performance of underparameterized models tends to be variance dominated, so classical experimental design focuses on variance reduction, while the predictive performance of overparameterized models can also be, as is shown in this paper, bias dominated or of mixed nature. In this paper we propose a design strategy that is well suited for overparameterized regression and interpolation, and we demonstrate the applicability of our method in the context of deep learning by proposing a new algorithm for single shot deep active learning.
