论文标题
学习潜在空间能量模型的自适应多阶段密度比估计
Adaptive Multi-stage Density Ratio Estimation for Learning Latent Space Energy-based Model
论文作者
论文摘要
本文研究了发电机模型潜在空间中基于学习能量的模型（EBM）的基本问题。学习此类先前的模型通常需要运行昂贵的Markov Chain Monte Carlo（MCMC）。取而代之的是，我们建议使用噪声对比估计（NCE），通过潜在的先前密度和潜在的后验密度之间的密度比估计来区分EBM。但是，如果两个密度之间的差距很大，则NCE通常无法准确估计这种密度比。为了有效解决此问题并学习更具表现力的先验模型，我们开发了自适应多阶段密度比估计，该估计将估计分为多个阶段，并依次和适应性地学习密度比的不同阶段。可以使用前阶段估计的比率逐渐学习潜在的先验模型，以便最终的潜在空间EBM先验可以通过不同阶段的比率产物自然形成。所提出的方法比现有基线可以提供信息，并且可以有效地培训。我们的实验表明，在图像产生和重建以及异常检测中表现出色。
This paper studies the fundamental problem of learning energy-based model (EBM) in the latent space of the generator model. Learning such prior model typically requires running costly Markov Chain Monte Carlo (MCMC). Instead, we propose to use noise contrastive estimation (NCE) to discriminatively learn the EBM through density ratio estimation between the latent prior density and latent posterior density. However, the NCE typically fails to accurately estimate such density ratio given large gap between two densities. To effectively tackle this issue and learn more expressive prior models, we develop the adaptive multi-stage density ratio estimation which breaks the estimation into multiple stages and learn different stages of density ratio sequentially and adaptively. The latent prior model can be gradually learned using ratio estimated in previous stage so that the final latent space EBM prior can be naturally formed by product of ratios in different stages. The proposed method enables informative and much sharper prior than existing baselines, and can be trained efficiently. Our experiments demonstrate strong performances in image generation and reconstruction as well as anomaly detection.
