论文标题
通过局部线性化改善贝叶斯神经网的预测
Improving predictions of Bayesian neural nets via local linearization
论文作者
论文摘要
通用的高斯 - 纽顿（GGN）近似通常用于通过用一阶导数的产物代替二阶导数来使实用的贝叶斯深度学习方法可扩展。在本文中，我们认为GGN近似应理解为基础贝叶斯神经网络（BNN）的局部线性化，该网络将BNN变成了广义线性模型（GLM）。因为我们将此线性化模型用于后推理，所以我们还应该预测使用此修改的模型而不是原始模型。我们将这种修改后的预测性称为“ GLM预测”，并表明它有效解决了拉普拉斯近似的常见不足问题。它将这种静脉的先前结果扩展到一般的可能性，并具有等效的高斯工艺公式，这可以实现功能空间中BNN的替代推理方案。我们证明了方法对几个标准分类数据集以及分布式检测的有效性。我们在https://github.com/aleximmer/bnn-predictions上提供实施。
The generalized Gauss-Newton (GGN) approximation is often used to make practical Bayesian deep learning approaches scalable by replacing a second order derivative with a product of first order derivatives. In this paper we argue that the GGN approximation should be understood as a local linearization of the underlying Bayesian neural network (BNN), which turns the BNN into a generalized linear model (GLM). Because we use this linearized model for posterior inference, we should also predict using this modified model instead of the original one. We refer to this modified predictive as "GLM predictive" and show that it effectively resolves common underfitting problems of the Laplace approximation. It extends previous results in this vein to general likelihoods and has an equivalent Gaussian process formulation, which enables alternative inference schemes for BNNs in function space. We demonstrate the effectiveness of our approach on several standard classification datasets as well as on out-of-distribution detection. We provide an implementation at https://github.com/AlexImmer/BNN-predictions.
