论文标题
灵敏度辅助交替方向乘数的分布式优化和统计学习的方法
Sensitivity Assisted Alternating Directions Method of Multipliers for Distributed Optimization and Statistical Learning
论文作者
论文摘要
本文考虑了使用乘数的交替说明方法（ADMM）考虑分布式模型拟合的问题。 ADMM通常通过对数据样本进行分区，将学习问题分为几个较小的子问题。可以通过一组由主节点协调的工人计算节点并行解决不同的子问题，并反复求解子问题直到收敛。在每次迭代中，工人节点必须解决一个凸优化问题，该问题的难度随问题的大小而增加。在本文中，我们提出了一种灵敏度辅助的ADMM算法，该算法利用了参数灵敏度，以便可以使用切向预测指标近似子问题解决方案，从而减轻计算负担来计算一个线性线性。我们研究了提出的敏感性辅助ADMM算法的收敛性。在非线性参数估计问题以及多层感知器学习问题上说明了算法的数值性能。
This paper considers the problem of distributed model fitting using the alternating directions method of multipliers (ADMM). ADMM splits the learning problem into several smaller subproblems, usually by partitioning the data samples. The different subproblems can be solved in parallel by a set of worker computing nodes coordinated by a master node, and the subproblems are repeatedly solved until convergence. At each iteration, the worker nodes must solve a convex optimization problem whose difficulty increases with the size of the problem. In this paper, we propose a sensitivity-assisted ADMM algorithm that leverages the parametric sensitivities such that the subproblems solutions can be approximated using a tangential predictor, thus easing the computational burden to computing one linear solve. We study the convergence properties of the proposed sensitivity-assisted ADMM algorithm. The numerical performance of the algorithm is illustrated on a nonlinear parameter estimation problem, and a multilayer perceptron learning problem.
