论文标题
具有随机特征的学习的精确表现分析
A Precise Performance Analysis of Learning with Random Features
论文作者
论文摘要
我们研究使用随机特征模型学习未知功能的问题。我们的主要贡献是对高斯数据的此类学习问题的确切渐近分析。在特征矩阵的轻度规律性条件下，我们提供了渐近训练和泛化错误的确切表征，在参数较低和参数过度分析方面有效。本文中提供的分析为特征矩阵，激活函数和凸丢失函数的一般家庭提供。数值结果验证了我们的理论预测，表明我们的渐近发现与所考虑的学习问题的实际表现非常吻合，即使在适度的维度上也是如此。此外，它们揭示了在学习中“双重下降现象”缓解“双血后裔现象”中的正则化，损耗函数和激活函数所起的重要作用。
We study the problem of learning an unknown function using random feature models. Our main contribution is an exact asymptotic analysis of such learning problems with Gaussian data. Under mild regularity conditions for the feature matrix, we provide an exact characterization of the asymptotic training and generalization errors, valid in both the under-parameterized and over-parameterized regimes. The analysis presented in this paper holds for general families of feature matrices, activation functions, and convex loss functions. Numerical results validate our theoretical predictions, showing that our asymptotic findings are in excellent agreement with the actual performance of the considered learning problem, even in moderate dimensions. Moreover, they reveal an important role played by the regularization, the loss function and the activation function in the mitigation of the "double descent phenomenon" in learning.
