论文标题
样本加权作为生成对抗网络中模式崩溃的解释
Sample weighting as an explanation for mode collapse in generative adversarial networks
论文作者
论文摘要
引入了具有逻辑最小成本表述的生成对抗网络，通常由于饱和而无法训练，并且不饱和重新制定。在解决饱和问题的同时，NS-GAN还将发电机的样本加权颠倒，在更新参数时隐含地将重点从得分较高的样本转移到了较低得分的样本。我们提出了理论和经验结果，这表明这使得NS-GAN容易降低模式。我们设计了MM-NSAT，可以保留MM-GAN样品加权，同时通过重新缩放MM-GAN MiniBatch梯度避免饱和，以使其幅度近似于NS-GAN的梯度幅度。 MM-NSAT具有质量不同的训练动力学，并且在MNIST和CIFAR-10上，在模式覆盖，稳定性和FID方面，它更强。尽管与LS-GAN和HEINGE-GAN配方相比，MM-NSAT的经验结果也很有希望且有利，但我们的主要贡献是展示NS-GAN的样品加权如何以及为什么会导致模式下降和训练崩溃。
Generative adversarial networks were introduced with a logistic MiniMax cost formulation, which normally fails to train due to saturation, and a Non-Saturating reformulation. While addressing the saturation problem, NS-GAN also inverts the generator's sample weighting, implicitly shifting emphasis from higher-scoring to lower-scoring samples when updating parameters. We present both theory and empirical results suggesting that this makes NS-GAN prone to mode dropping. We design MM-nsat, which preserves MM-GAN sample weighting while avoiding saturation by rescaling the MM-GAN minibatch gradient such that its magnitude approximates NS-GAN's gradient magnitude. MM-nsat has qualitatively different training dynamics, and on MNIST and CIFAR-10 it is stronger in terms of mode coverage, stability and FID. While the empirical results for MM-nsat are promising and favorable also in comparison with the LS-GAN and Hinge-GAN formulations, our main contribution is to show how and why NS-GAN's sample weighting causes mode dropping and training collapse.
