论文标题
gan训练中的对抗模式崩溃：使用Hessian特征值进行的经验分析
Combating Mode Collapse in GAN training: An Empirical Analysis using Hessian Eigenvalues
论文作者
论文摘要
生成对抗网络（GAN）提供了图像生成的最新结果。但是，尽管有如此强大的训练，但他们仍然非常具有挑战性。这特别是由其高度非凸优化空间引起的，导致许多不稳定性。其中，模式崩溃是最艰巨的崩溃之一。当模型只能符合数据分布的几种模式，而忽略大多数数据时，就会发生这种不良事件。在这项工作中，我们使用二阶梯度信息对抗模式崩溃。为此，我们通过其Hessian特征值分析了损失表面，并表明模式崩溃与朝向最小值的收敛有关。特别是，我们观察到$ g $的特征值与模式崩溃的发生直接相关。最后，在这些发现的激励下，我们设计了一种称为Nudged-Adam（Nugan）的新优化算法，该算法使用光谱信息来克服模式崩溃，从而导致经验上更稳定的收敛性能。
Generative adversarial networks (GANs) provide state-of-the-art results in image generation. However, despite being so powerful, they still remain very challenging to train. This is in particular caused by their highly non-convex optimization space leading to a number of instabilities. Among them, mode collapse stands out as one of the most daunting ones. This undesirable event occurs when the model can only fit a few modes of the data distribution, while ignoring the majority of them. In this work, we combat mode collapse using second-order gradient information. To do so, we analyse the loss surface through its Hessian eigenvalues, and show that mode collapse is related to the convergence towards sharp minima. In particular, we observe how the eigenvalues of the $G$ are directly correlated with the occurrence of mode collapse. Finally, motivated by these findings, we design a new optimization algorithm called nudged-Adam (NuGAN) that uses spectral information to overcome mode collapse, leading to empirically more stable convergence properties.
