论文标题
神经网络是凸正规化器：两层网络的精确多项式凸优化公式
Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-layer Networks
论文作者
论文摘要
我们根据单个凸面程序来开发具有整流线性单元（RELUS）的训练两层神经网络的确切表示形式，该程序数量在训练样本的数量和隐藏的神经元的数量中多项式。我们的理论利用了半无限双重性和最低规范正则化。我们证明，经过标准重量衰减训练的Relu网络等效于阻止$ \ ell_1 $惩罚的凸模型。此外，我们表明某些标准卷积线性网络是等效的半明确程序，可以将其简化为$ \ ell_1 $正规化线性模型，在多项式尺寸的离散傅立叶功能空间中。
We develop exact representations of training two-layer neural networks with rectified linear units (ReLUs) in terms of a single convex program with number of variables polynomial in the number of training samples and the number of hidden neurons. Our theory utilizes semi-infinite duality and minimum norm regularization. We show that ReLU networks trained with standard weight decay are equivalent to block $\ell_1$ penalized convex models. Moreover, we show that certain standard convolutional linear networks are equivalent semi-definite programs which can be simplified to $\ell_1$ regularized linear models in a polynomial sized discrete Fourier feature space.
