论文标题
数据增强作为功能操纵
Data Augmentation as Feature Manipulation
论文作者
论文摘要
数据增强是机器学习管道的基石，但其理论基础尚不清楚。它只是人为增加数据集大小的一种方法吗？还是鼓励模型满足某些不变性？在这项工作中，我们考虑了另一个角度，我们研究了数据增强对学习过程动态的影响。我们发现，数据扩展可以改变各种功能的相对重要性，从而有效地使某些信息丰富，但很难学习在学习过程中更有可能被捕获。重要的是，我们表明，对于非线性模型，例如神经网络，这种效果更为明显。我们的主要贡献是对Allen-Zhu和Li [2020]的最近提出的多视图数据模型中两层卷积神经网络的学习动态数据的详细分析。我们通过进一步的实验证据对这一分析进行补充，证明数据增加可以视为特征操纵。
Data augmentation is a cornerstone of the machine learning pipeline, yet its theoretical underpinnings remain unclear. Is it merely a way to artificially augment the data set size? Or is it about encouraging the model to satisfy certain invariance? In this work we consider another angle, and we study the effect of data augmentation on the dynamic of the learning process. We find that data augmentation can alter the relative importance of various features, effectively making certain informative but hard to learn features more likely to be captured in the learning process. Importantly, we show that this effect is more pronounced for non-linear models, such as neural networks. Our main contribution is a detailed analysis of data augmentation on the learning dynamic for a two layer convolutional neural network in the recently proposed multi-view data model by Allen-Zhu and Li [2020]. We complement this analysis with further experimental evidence that data augmentation can be viewed as feature manipulation.
