论文标题
随机平均模型方法
Stochastic Average Model Methods
论文作者
论文摘要
我们考虑有限和最小化问题的解决方案，例如在非线性最小二乘中出现的问题或一般的经验风险最小化问题。我们是受到汇总函数在计算上昂贵的问题的动机，并且在每种优化方法的每种迭代中评估所有汇总可能是不可取的。我们介绍了由随机平均梯度方法启发的随机平均模型（SAM）方法的想法。 SAM方法根据组件函数上的离散概率分布，在信任区域方法的每次迭代中示例组件函数；该分布旨在最大程度地减少所得随机模型方差的上限。我们提出了有关实施变体的有希望的数值结果，该变体扩展了基于无导数模型的信任区域求解器，我们将其命名为Sam-pounders。
We consider the solution of finite-sum minimization problems, such as those appearing in nonlinear least-squares or general empirical risk minimization problems. We are motivated by problems in which the summand functions are computationally expensive and evaluating all summands on every iteration of an optimization method may be undesirable. We present the idea of stochastic average model (SAM) methods, inspired by stochastic average gradient methods. SAM methods sample component functions on each iteration of a trust-region method according to a discrete probability distribution on component functions; the distribution is designed to minimize an upper bound on the variance of the resulting stochastic model. We present promising numerical results concerning an implemented variant extending the derivative-free model-based trust-region solver POUNDERS, which we name SAM-POUNDERS.
