论文标题
通过有条件信息测量的快速损失界限，并应用于神经网络
Fast-Rate Loss Bounds via Conditional Information Measures with Applications to Neural Networks
论文作者
论文摘要
我们提出了一个框架，以在有限损失函数的情况下得出关于随机学习算法的测试损失的界限。从Steinke＆Zakynthinou（2020）绘制，该框架导致界限取决于输出假设和训练集选择之间的条件信息密度，鉴于形成了训练集的较大数据样本。此外，对于Pac-bayesian和单draw设置，界限与平均测试损失以及其尾巴概率有关。如果条件信息密度以训练集的尺寸$ n $统一界定，我们的边界衰减为$ 1/n $。这与涉及文献中可用的条件信息度量的尾巴范围相反，文献中的良性$ 1/\ sqrt {n} $依赖性。我们通过证明它们导致对测试损失的估计值，通过对MNIST和时尚训练的某些神经网络体系结构进行测试损失的估计来证明我们的尾巴界限的实用性。
We present a framework to derive bounds on the test loss of randomized learning algorithms for the case of bounded loss functions. Drawing from Steinke & Zakynthinou (2020), this framework leads to bounds that depend on the conditional information density between the the output hypothesis and the choice of the training set, given a larger set of data samples from which the training set is formed. Furthermore, the bounds pertain to the average test loss as well as to its tail probability, both for the PAC-Bayesian and the single-draw settings. If the conditional information density is bounded uniformly in the size $n$ of the training set, our bounds decay as $1/n$. This is in contrast with the tail bounds involving conditional information measures available in the literature, which have a less benign $1/\sqrt{n}$ dependence. We demonstrate the usefulness of our tail bounds by showing that they lead to nonvacuous estimates of the test loss achievable with some neural network architectures trained on MNIST and Fashion-MNIST.
