论文标题
Riemannian随机方差降低的立方体牛顿方法用于亚曼叶量优化
Riemannian Stochastic Variance-Reduced Cubic Regularized Newton Method for Submanifold Optimization
论文作者
论文摘要
我们提出了一种随机方差降低的立方体牛顿算法，以优化在欧几里得空间的Riemannian Submanifold上优化有限的和问题。所提出的算法需要在每个时期的开头进行完整的梯度和Hessian更新，同时在每个时期内的迭代中执行随机方差减少更新。 $ O（ε^{ - 3/2}）$的迭代复杂性要获得$（ε，\sqrtε）$ - 二阶固定点，即，与Riemannian梯度规范在$ε$和Riemann permimim demmimim demmimim formimim formimum byed by the Riemannian梯度规范上限制为$ - 欧几里得空间。此外，本文提出了对算法的计算更具吸引力的修改，该修改仅需要具有相同迭代复杂性的立方正规化牛顿子问题的不精确解决方案。在两项数值研究中，评估了所提出的算法并将其与其他三种riemannian二阶方法进行比较，以估算多元T分布的反向矩阵，这些矩阵在对称正定矩阵的歧管上，并估算球体歧管上线性分类的参数。
We propose a stochastic variance-reduced cubic regularized Newton algorithm to optimize the finite-sum problem over a Riemannian submanifold of the Euclidean space. The proposed algorithm requires a full gradient and Hessian update at the beginning of each epoch while it performs stochastic variance-reduced updates in the iterations within each epoch. The iteration complexity of $O(ε^{-3/2})$ to obtain an $(ε,\sqrtε)$-second-order stationary point, i.e., a point with the Riemannian gradient norm upper bounded by $ε$ and minimum eigenvalue of Riemannian Hessian lower bounded by $-\sqrtε$, is established when the manifold is embedded in the Euclidean space. Furthermore, the paper proposes a computationally more appealing modification of the algorithm which only requires an inexact solution of the cubic regularized Newton subproblem with the same iteration complexity. The proposed algorithm is evaluated and compared with three other Riemannian second-order methods over two numerical studies on estimating the inverse scale matrix of the multivariate t-distribution on the manifold of symmetric positive definite matrices and estimating the parameter of a linear classifier on the Sphere manifold.
