论文标题
微小：从负曲率检测到单调性能
MINRES: From Negative Curvature Detection to Monotonicity Properties
论文作者
论文摘要
长期以来，共轭梯度法（CG）一直是大规模非convex优化的二阶算法的内部训练。突出的示例包括基于线路搜索的算法，例如牛顿-CG，以及基于信任区域框架的算法，例如CG-Steihaug。这主要归功于CG的几个有利属性，包括某些单调性属性及其固有的检测负曲率方向的能力，这可能在非convex优化中出现。尽管事实是，当涉及到真正的对称但潜在不确定的矩阵时，迭代选择方法可以说是著名的最小残留方法（minres）方法。但是，在这种设置中，微小值暗示的相似属性的有限理解限制了其在非凸优化算法中的适用性。我们建立了微小的几种此类非平凡特性，包括某些有用的单调性以及固有的检测负曲率方向的能力。这些特性允许对所有使用CG作为其子问题求解器的牛顿型非convex优化算法，将minres视为CG的潜在替代品。
The conjugate gradient method (CG) has long been the workhorse for inner-iterations of second-order algorithms for large-scale nonconvex optimization. Prominent examples include line-search based algorithms, e.g., Newton-CG, and those based on a trust-region framework, e.g., CG-Steihaug. This is mainly thanks to CG's several favorable properties, including certain monotonicity properties and its inherent ability to detect negative curvature directions, which can arise in nonconvex optimization. This is despite the fact that the iterative method-of-choice when it comes to real symmetric but potentially indefinite matrices is arguably the celebrated minimal residual (MINRES) method. However, limited understanding of similar properties implied by MINRES in such settings has restricted its applicability within nonconvex optimization algorithms. We establish several such nontrivial properties of MINRES, including certain useful monotonicity as well as an inherent ability to detect negative curvature directions. These properties allow MINRES to be considered as a potentially superior alternative to CG for all Newton-type nonconvex optimization algorithms that employ CG as their subproblem solver.
