论文标题
学习通过动态模式分解进行优化
Learning to Optimize with Dynamic Mode Decomposition
论文作者
论文摘要
设计更快的优化算法具有不断增长的兴趣。近年来，学习学习如何优化如何优化的方法表现出非常令人鼓舞的结果。当前的方法通常不能有效地包括训练过程中优化过程的动力学。他们要么完全忽略它，要么仅隐式地假设孤立参数的动力学。在本文中，我们展示了如何利用动态模式分解方法来提取有关优化动力学的信息特征。通过采用这些功能，我们表明我们的学习优化器可以更好地概括，从而简单地看不到优化问题。在多个任务上说明了改进的概括，其中训练一个神经网络上的优化器将其推广到不同的架构和不同的数据集。
Designing faster optimization algorithms is of ever-growing interest. In recent years, learning to learn methods that learn how to optimize demonstrated very encouraging results. Current approaches usually do not effectively include the dynamics of the optimization process during training. They either omit it entirely or only implicitly assume the dynamics of an isolated parameter. In this paper, we show how to utilize the dynamic mode decomposition method for extracting informative features about optimization dynamics. By employing those features, we show that our learned optimizer generalizes much better to unseen optimization problems in short. The improved generalization is illustrated on multiple tasks where training the optimizer on one neural network generalizes to different architectures and distinct datasets.
