论文标题
弥合基于联合能量的模型的性能差距
Towards Bridging the Performance Gaps of Joint Energy-based Models
论文作者
论文摘要
我们可以在单个网络中训练混合歧视生成模型吗？最近在肯定中回答了这个问题，引入了基于联合能量的模型（JEM）的领域，该模型（JEM）同时达到了高分类的准确性和图像生成质量。尽管有最近的进步，但仍存在两个性能差距：标准软智能分类器的准确性差距，以及最先进的生成模型的发电质量差距。在本文中，我们介绍了各种培训技术，以弥合JEM的准确性差距和一代质量差距。 1）我们结合了最近提出的清晰度最小化（SAM）框架来训练JEM，从而促进了能量景观的平滑度和JEM的普遍性。 2）我们将数据扩展排除在JEM的最大似然估计管道中，并减轻数据增强对图像生成质量的负面影响。在多个数据集上进行的广泛实验表明，我们的Sada-Jem在图像分类，图像产生，校准，分布外检测和对抗性鲁棒性方面实现了最先进的表现，并优于JEM JEM。我们的代码可在https://github.com/sndnyang/sadajem上找到。
Can we train a hybrid discriminative-generative model within a single network? This question has recently been answered in the affirmative, introducing the field of Joint Energy-based Model (JEM), which achieves high classification accuracy and image generation quality simultaneously. Despite recent advances, there remain two performance gaps: the accuracy gap to the standard softmax classifier, and the generation quality gap to state-of-the-art generative models. In this paper, we introduce a variety of training techniques to bridge the accuracy gap and the generation quality gap of JEM. 1) We incorporate a recently proposed sharpness-aware minimization (SAM) framework to train JEM, which promotes the energy landscape smoothness and the generalizability of JEM. 2) We exclude data augmentation from the maximum likelihood estimate pipeline of JEM, and mitigate the negative impact of data augmentation to image generation quality. Extensive experiments on multiple datasets demonstrate that our SADA-JEM achieves state-of-the-art performances and outperforms JEM in image classification, image generation, calibration, out-of-distribution detection and adversarial robustness by a notable margin. Our code is available at https://github.com/sndnyang/SADAJEM.
