论文标题
平均场Langevin动力学：指数收敛和退火
Mean-Field Langevin Dynamics: Exponential Convergence and Annealing
论文作者
论文摘要
嘈杂的粒子梯度下降（NPGD）是一种算法，可在包括熵项在内的度量空间中最小化凸功能。在许多粒子极限中，该算法由平均场兰格文动力学描述 - 非线性漂移的langevin动力学的概括 - 这是我们的主要研究对象。以前的工作已经通过非量化参数表明了它与独特的最小化器的融合。我们证明，这种动力学以指数速率收敛，假设某个log-sobolev不平等的家族存在。例如，该假设可将某些两层神经网络的风险最小化，其中NPGD等于标准的噪声梯度下降。我们还研究了退火的动力学，并表明，对于以对数速率噪声衰减，动力学的价值会收敛于未注册的目标函数的全局最小化器。
Noisy particle gradient descent (NPGD) is an algorithm to minimize convex functions over the space of measures that include an entropy term. In the many-particle limit, this algorithm is described by a Mean-Field Langevin dynamics - a generalization of the Langevin dynamics with a non-linear drift - which is our main object of study. Previous work have shown its convergence to the unique minimizer via non-quantitative arguments. We prove that this dynamics converges at an exponential rate, under the assumption that a certain family of Log-Sobolev inequalities holds. This assumption holds for instance for the minimization of the risk of certain two-layer neural networks, where NPGD is equivalent to standard noisy gradient descent. We also study the annealed dynamics, and show that for a noise decaying at a logarithmic rate, the dynamics converges in value to the global minimizer of the unregularized objective function.
