论文标题
无限宽的深神经网络的稳定行为
Stable behaviour of infinitely wide deep neural networks
论文作者
论文摘要
我们认为重量和偏见是独立的，并且分布相同作为对称中心稳定分布，我们考虑了完全连接的进料深度神经网络（NNS）。然后，我们表明，在权重的适当缩放下，NN的无限宽极限是一个随机过程，其有限维分布是多元稳定分布。限制过程称为稳定过程，它概括了最近作为NNS的无限范围获得的高斯过程（Matthews at al。，2018b）。稳定过程的参数可以通过网络层的明确递归计算。我们的结果有助于完全连接的进料深度NNS的理论，并为扩大依赖高斯无限范围的最新研究范围铺平了道路。
We consider fully connected feed-forward deep neural networks (NNs) where weights and biases are independent and identically distributed as symmetric centered stable distributions. Then, we show that the infinite wide limit of the NN, under suitable scaling on the weights, is a stochastic process whose finite-dimensional distributions are multivariate stable distributions. The limiting process is referred to as the stable process, and it generalizes the class of Gaussian processes recently obtained as infinite wide limits of NNs (Matthews at al., 2018b). Parameters of the stable process can be computed via an explicit recursion over the layers of the network. Our result contributes to the theory of fully connected feed-forward deep NNs, and it paves the way to expand recent lines of research that rely on Gaussian infinite wide limits.
