论文标题
在具有多项式激活的宽网络的渐近学上
On the asymptotics of wide networks with polynomial activations
论文作者
论文摘要
我们考虑了一个现有的猜想，该猜想是针对较大宽度极限的神经网络的渐近行为。此猜想随之而来的结果包括在随机梯度下降过程中宽网络行为的紧密界限，以及其有限宽度动力学的推导。我们证明了具有多项式激活函数的深网的猜想，从而大大扩展了这些结果的有效性。最后，我们指出具有分析性（和非线性）激活函数网络的渐近行为以及具有分段线性激活（例如Relu）的渐近行为。
We consider an existing conjecture addressing the asymptotic behavior of neural networks in the large width limit. The results that follow from this conjecture include tight bounds on the behavior of wide networks during stochastic gradient descent, and a derivation of their finite-width dynamics. We prove the conjecture for deep networks with polynomial activation functions, greatly extending the validity of these results. Finally, we point out a difference in the asymptotic behavior of networks with analytic (and non-linear) activation functions and those with piecewise-linear activations such as ReLU.
