论文标题
基于决策边界的复杂性，对深神经网络的普遍性分析
Analysis of Generalizability of Deep Neural Networks Based on the Complexity of Decision Boundary
论文作者
论文摘要
对于监督学习模型，对概括能力（概括性）的分析至关重要，因为概括性表达了模型对看不见的数据的表现。传统的概括方法（例如VC维度）不适用于深神经网络（DNN）模型。因此，需要新的理论来解释DNN的普遍性。在这项研究中，我们假设具有更简单的决策边界的DNN可以通过Parsimony定律（Occam的剃须刀）更好地概括性。我们创建决策边界复杂性（DBC）得分，以定义和测量DNNS决策边界的复杂性。 DBC分数的想法是在决策边界或附近生成数据点（称为对抗性示例）。然后，我们的新方法使用这些数据的特征值的熵来衡量边界的复杂性。该方法在高维数据方面同样效果很好。我们使用培训数据和训练有素的模型来计算DBC分数。而且，模型可推广性的基础真实是其测试准确性。基于DBC评分的实验已经验证了我们的假设。显示DBC提供了一种有效的方法来衡量决策边界的复杂性，并提供了DNN的普遍性的定量度量。
For supervised learning models, the analysis of generalization ability (generalizability) is vital because the generalizability expresses how well a model will perform on unseen data. Traditional generalization methods, such as the VC dimension, do not apply to deep neural network (DNN) models. Thus, new theories to explain the generalizability of DNNs are required. In this study, we hypothesize that the DNN with a simpler decision boundary has better generalizability by the law of parsimony (Occam's Razor). We create the decision boundary complexity (DBC) score to define and measure the complexity of decision boundary of DNNs. The idea of the DBC score is to generate data points (called adversarial examples) on or near the decision boundary. Our new approach then measures the complexity of the boundary using the entropy of eigenvalues of these data. The method works equally well for high-dimensional data. We use training data and the trained model to compute the DBC score. And, the ground truth for model's generalizability is its test accuracy. Experiments based on the DBC score have verified our hypothesis. The DBC is shown to provide an effective method to measure the complexity of a decision boundary and gives a quantitative measure of the generalizability of DNNs.
