论文标题
如何填充最佳集合？人口梯度下降，多样性无害
How to Fill the Optimum Set? Population Gradient Descent with Harmless Diversity
论文作者
论文摘要
尽管传统的优化方法着重于找到单个最佳解决方案，但是现代机器学习问题中的大多数客观功能，尤其是在深度学习中，通常具有多个或无限数量的Optima。因此，考虑在最佳目标函数集中找到一组不同点的问题是有用的。在这项工作中，我们将此问题框架为双层优化问题，即在主要损失函数的最佳集合中最大化多样性分数，并使用简单的人群梯度下降框架来解决它，该框架迭代地更新点以最大程度地提高多样性得分，以不损害主损失优化的方式。我们证明，我们的方法可以在各种应用程序上有效地生成多种解决方案，包括文本到图像生成，文本到网格的生成，分子构象产生和集成神经网络培训。
Although traditional optimization methods focus on finding a single optimal solution, most objective functions in modern machine learning problems, especially those in deep learning, often have multiple or infinite numbers of optima. Therefore, it is useful to consider the problem of finding a set of diverse points in the optimum set of an objective function. In this work, we frame this problem as a bi-level optimization problem of maximizing a diversity score inside the optimum set of the main loss function, and solve it with a simple population gradient descent framework that iteratively updates the points to maximize the diversity score in a fashion that does not hurt the optimization of the main loss. We demonstrate that our method can efficiently generate diverse solutions on a variety of applications, including text-to-image generation, text-to-mesh generation, molecular conformation generation and ensemble neural network training.
