论文标题
Black-Box VI的进步：标准化流量，重要性加权和优化
Advances in Black-Box VI: Normalizing Flows, Importance Weighting, and Optimization
论文作者
论文摘要
最近的研究已经看到了与Black-Box VI相关的几项进展，但是当前自动后推理的状态尚不清楚。这样的进步之一是使用标准化流以定义深层可变模型的柔性后密度。另一个方向是整合蒙特卡洛方法以实现两个目的的方法。首先，要获得更严格的变异目标，其次是通过抽样来定义丰富的变异家族。但是，对于黑盒VI，流量和变异的蒙特卡洛方法仍然相对尚未探索。此外，在务实的方面，有几种优化考虑因素，例如阶梯尺寸方案，参数初始化和梯度估计器的选择，在现有文献中没有明确的指导。在本文中，我们假设通过众多算法组件的仔细组合来最好地解决黑盒VI。我们在Stan模型库的30个模型的基准上评估了与优化，流量和蒙特卡罗方法有关的组件。这些算法组件的组合显着提高了最新的“开箱即用”变化推断。
Recent research has seen several advances relevant to black-box VI, but the current state of automatic posterior inference is unclear. One such advance is the use of normalizing flows to define flexible posterior densities for deep latent variable models. Another direction is the integration of Monte-Carlo methods to serve two purposes; first, to obtain tighter variational objectives for optimization, and second, to define enriched variational families through sampling. However, both flows and variational Monte-Carlo methods remain relatively unexplored for black-box VI. Moreover, on a pragmatic front, there are several optimization considerations like step-size scheme, parameter initialization, and choice of gradient estimators, for which there are no clear guidance in the existing literature. In this paper, we postulate that black-box VI is best addressed through a careful combination of numerous algorithmic components. We evaluate components relating to optimization, flows, and Monte-Carlo methods on a benchmark of 30 models from the Stan model library. The combination of these algorithmic components significantly advances the state-of-the-art "out of the box" variational inference.
