论文标题
通过基于插值的投影及其在深度学习中的应用，凸优化
Convex Optimization with an Interpolation-based Projection and its Application to Deep Learning
论文作者
论文摘要
凸优化器将许多应用称为深神经体系结构中的可区分层。这些凸面层的一种应用是将点点点为凸集。但是，这些凸面层的前进和向后通过比典型的神经网络的计算层要昂贵得多。我们在本文中研究了不精确但廉价投影是否可以将下降算法驱动到最佳。具体而言，我们提出了一个基于插值的投影，该投影在计算上便宜且易于计算给定凸，域定义，功能。然后，我们提出了一种遵循目标组成和投影梯度的优化算法，并证明了其对线性目标以及任意凸和Lipschitz域定义不平等约束的收敛性。除了理论贡献外，我们从经验上证明了与神经网络一起在增强学习和监督学习环境中与神经网络一起使用时的插值投影的实际利益。
Convex optimizers have known many applications as differentiable layers within deep neural architectures. One application of these convex layers is to project points into a convex set. However, both forward and backward passes of these convex layers are significantly more expensive to compute than those of a typical neural network. We investigate in this paper whether an inexact, but cheaper projection, can drive a descent algorithm to an optimum. Specifically, we propose an interpolation-based projection that is computationally cheap and easy to compute given a convex, domain defining, function. We then propose an optimization algorithm that follows the gradient of the composition of the objective and the projection and prove its convergence for linear objectives and arbitrary convex and Lipschitz domain defining inequality constraints. In addition to the theoretical contributions, we demonstrate empirically the practical interest of the interpolation projection when used in conjunction with neural networks in a reinforcement learning and a supervised learning setting.
