论文标题
BFE和ADABFE：一种新的学习率自动化方法，用于随机优化
BFE and AdaBFE: A New Approach in Learning Rate Automation for Stochastic Optimization
论文作者
论文摘要
在本文中，提出了一种新的基于梯度的优化方法，可以自动调整学习率。这种方法可以应用于设计非自适应学习率和自适应学习率。首先，我将介绍非自适应学习率优化方法：二进制前进探索（BFE），然后可以开发相应的自适应人均学习率方法：自适应BFE（ADABFE）。这种方法可能是一种基于当前非自适应学习率方法（例如SGD，动量，Nesterov和自适应学习率方法，例如Adagrad，Adadelta，Adam ...开发这种方法的目的不是要击败其他方法的基准，而只是提供不同的观点来优化梯度下降方法，尽管以下各节将进行一些比较研究。预计这种方法将是启发式方法或激发研究人员改善基于梯度的优化以及以前的方法。
In this paper, a new gradient-based optimization approach by automatically adjusting the learning rate is proposed. This approach can be applied to design non-adaptive learning rate and adaptive learning rate. Firstly, I will introduce the non-adaptive learning rate optimization method: Binary Forward Exploration (BFE), and then the corresponding adaptive per-parameter learning rate method: Adaptive BFE (AdaBFE) is possible to be developed. This approach could be an alternative method to optimize the learning rate based on the stochastic gradient descent (SGD) algorithm besides the current non-adaptive learning rate methods e.g. SGD, momentum, Nesterov and the adaptive learning rate methods e.g. AdaGrad, AdaDelta, Adam... The purpose to develop this approach is not to beat the benchmark of other methods but just to provide a different perspective to optimize the gradient descent method, although some comparative study with previous methods will be made in the following sections. This approach is expected to be heuristic or inspire researchers to improve gradient-based optimization combined with previous methods.
