论文标题
使用小波的合并损失的自然梯度
Natural Gradient for Combined Loss Using Wavelets
论文作者
论文摘要
自然梯度已被广泛用于优化损失功能在概率空间上的优化，其中重要的例子，例如用于Kullback-Leibler Divergence的Fisher-Rao梯度下降，用于运输相关功能的Wasserstein梯度下降，而Mahalanobis梯度梯度梯度梯度梯度梯度梯度梯度下降。本说明认为损失是这些示例的凸线性组合的情况。我们提出了一种新的天然梯度算法，通过使用紧凑的小波将大约对对角线进行对角线化。包括数值结果以证明所提出的算法的效率。
Natural gradients have been widely used in optimization of loss functionals over probability space, with important examples such as Fisher-Rao gradient descent for Kullback-Leibler divergence, Wasserstein gradient descent for transport-related functionals, and Mahalanobis gradient descent for quadratic loss functionals. This note considers the situation in which the loss is a convex linear combination of these examples. We propose a new natural gradient algorithm by utilizing compactly supported wavelets to diagonalize approximately the Hessian of the combined loss. Numerical results are included to demonstrate the efficiency of the proposed algorithm.
