论文标题
随机方差使用Barzilai-Borwein技术降低梯度作为二阶信息
A Stochastic Variance Reduced Gradient using Barzilai-Borwein Techniques as Second Order Information
论文作者
论文摘要
在本文中，我们考虑通过结合目标函数的曲率信息来改善随机方差减少梯度（SVRG）方法。我们建议使用计算有效的Barzilai-Borwein（BB）方法来减少随机梯度的方差，并将其纳入SVRG。我们还将BB步骤大小合并为其变体。我们证明其线性收敛定理不仅适用于所提出的方法，还适用于SVRG的其他现有变体，并使用二阶信息。我们在基准数据集上进行了数值实验，并表明具有恒定步长的提议方法的性能优于现有方差减少的方法，这些方法对于某些测试问题。
In this paper, we consider to improve the stochastic variance reduce gradient (SVRG) method via incorporating the curvature information of the objective function. We propose to reduce the variance of stochastic gradients using the computationally efficient Barzilai-Borwein (BB) method by incorporating it into the SVRG. We also incorporate a BB-step size as its variant. We prove its linear convergence theorem that works not only for the proposed method but also for the other existing variants of SVRG with second-order information. We conduct the numerical experiments on the benchmark datasets and show that the proposed method with constant step size performs better than the existing variance reduced methods for some test problems.
