论文标题
学习relu网络达到高均匀的精度是棘手的
Learning ReLU networks to high uniform accuracy is intractable
论文作者
论文摘要
统计学习理论提供了在给定目标类别中提出的学习问题中所需的必要培训样本数量的界限。通常根据概括误差（即给定损耗函数的期望值）来衡量该准确性。但是，对于几种应用程序（例如在关键环境中或计算科学中的问题）中，从这个意义上讲，准确性是不够的。在这种情况下，人们希望在每个输入值（即相对于统一的规范）上保证高精度。在本文中，我们精确地量化了任何可以想象的培训算法所需的培训样本数量，以确保对包含（或组成）开处方架构的Relu神经网络的任何学习问题的给定均匀精度。我们证明，在非常一般的假设下，此任务的训练样本数量最少，在网络体系结构的深度和输入维度上成倍扩展。
Statistical learning theory provides bounds on the necessary number of training samples needed to reach a prescribed accuracy in a learning problem formulated over a given target class. This accuracy is typically measured in terms of a generalization error, that is, an expected value of a given loss function. However, for several applications -- for example in a security-critical context or for problems in the computational sciences -- accuracy in this sense is not sufficient. In such cases, one would like to have guarantees for high accuracy on every input value, that is, with respect to the uniform norm. In this paper we precisely quantify the number of training samples needed for any conceivable training algorithm to guarantee a given uniform accuracy on any learning problem formulated over target classes containing (or consisting of) ReLU neural networks of a prescribed architecture. We prove that, under very general assumptions, the minimal number of training samples for this task scales exponentially both in the depth and the input dimension of the network architecture.
