论文标题
在模型合奏中引起信息瓶颈的多样性
Diversity inducing Information Bottleneck in Model Ensembles
论文作者
论文摘要
尽管深度学习模型已经在许多视力任务上实现了最先进的表现，但对高维多模式数据的概括以及可靠的预测不确定性估计仍然是活跃的研究领域。包括贝叶斯神经网（BNN）在内的贝叶斯方法无法很好地扩展到现代计算机视觉任务，因为它们很难训练，并且在数据集偏移下的概括性较差。这促使需要有效的合奏，从而可以概括并提供可靠的不确定性估计。在本文中，我们针对通过鼓励预测多样性来产生神经网络有效合奏的问题。我们明确优化了多样性，以吸引对抗性损失，以学习随机潜在变量，从而在建模多模式数据所需的输出预测中获得多样性。我们在基准数据集上评估了我们的方法：MNIST，CIFAR100，Tinyimagenet和MIT Place 2，并与最有竞争力的基线相比，在数据分布和分布外检测中的变化下，分类准确性的提高显着提高。代码将在此URL https://github.com/rvl-lab-utoronto/dibs中发布
Although deep learning models have achieved state-of-the-art performance on a number of vision tasks, generalization over high dimensional multi-modal data, and reliable predictive uncertainty estimation are still active areas of research. Bayesian approaches including Bayesian Neural Nets (BNNs) do not scale well to modern computer vision tasks, as they are difficult to train, and have poor generalization under dataset-shift. This motivates the need for effective ensembles which can generalize and give reliable uncertainty estimates. In this paper, we target the problem of generating effective ensembles of neural networks by encouraging diversity in prediction. We explicitly optimize a diversity inducing adversarial loss for learning the stochastic latent variables and thereby obtain diversity in the output predictions necessary for modeling multi-modal data. We evaluate our method on benchmark datasets: MNIST, CIFAR100, TinyImageNet and MIT Places 2, and compared to the most competitive baselines show significant improvements in classification accuracy, under a shift in the data distribution and in out-of-distribution detection. Code will be released in this url https://github.com/rvl-lab-utoronto/dibs
