论文标题
掩盖不确定性估计
Masksembles for Uncertainty Estimation
论文作者
论文摘要
深度神经网络已经充分证明了他们的能力，但是估计预测的可靠性仍然具有挑战性。深层合奏被广泛认为是产生不确定性估计的最佳方法之一，但训练和评估非常昂贵。 Mc-Dropout是另一种流行的替代方案，它价格较低，但也较不可靠。我们的中心直觉是，有一个连续的集合样模型，其中Mc-Dropout和Deep Gemembles是极端的例子。第一个使用有效数量的高度相关模型，而第二个模型则依赖于有限数量的独立模型。 为了结合两者的好处，我们介绍了面具组合。 MaskSemble并没有像MC-Dropout中那样随机删除网络的一部分，而是依赖固定数量的二进制掩码，这些掩码是通过允许在单个模型之间更改相关性的方式进行参数化的。也就是说，通过控制蒙版及其密度之间的重叠，可以为手头任务选择最佳配置。这导致了一种易于实现的方法，其性能与合奏的成本相当一小部分。我们在两个广泛使用的数据集CIFAR10和Imagenet上实验验证了掩模。
Deep neural networks have amply demonstrated their prowess but estimating the reliability of their predictions remains challenging. Deep Ensembles are widely considered as being one of the best methods for generating uncertainty estimates but are very expensive to train and evaluate. MC-Dropout is another popular alternative, which is less expensive, but also less reliable. Our central intuition is that there is a continuous spectrum of ensemble-like models of which MC-Dropout and Deep Ensembles are extreme examples. The first uses an effectively infinite number of highly correlated models while the second relies on a finite number of independent models. To combine the benefits of both, we introduce Masksembles. Instead of randomly dropping parts of the network as in MC-dropout, Masksemble relies on a fixed number of binary masks, which are parameterized in a way that allows to change correlations between individual models. Namely, by controlling the overlap between the masks and their density one can choose the optimal configuration for the task at hand. This leads to a simple and easy to implement method with performance on par with Ensembles at a fraction of the cost. We experimentally validate Masksembles on two widely used datasets, CIFAR10 and ImageNet.
