论文标题
半自动精度和准确性分析的框架，以进行快速和严格的深度学习
A Framework for Semi-Automatic Precision and Accuracy Analysis for Fast and Rigorous Deep Learning
论文作者
论文摘要
深神经网络（DNN）代表渴望性能的应用。浮点（FP）和自定义浮点样算术可以满足这种饥饿感。尽管需要速度，但DNN中的推断似乎并不需要精确。许多论文在实验上观察到DNN可以以几乎荒谬的精度成功运行。 本文的目的是两个方面：首先，为为什么DNN的FP准确度以低FP精度保持较高的理论阐明。我们观察到，卷积步骤中相对准确性的损失是通过激活层回收的，该激活层的条件非常好。我们对DNN中精度和准确性之间的联系进行解释。 其次，本文为深度学习的推理阶段提供了用于半自动FP错误分析的软件框架。它与常见的Tensorflow/Keras模型兼容，它利用节俭的Python/C ++库将神经网络转换为C ++代码，以分析网络对精度的需求。这种严格的分析基于间隔和仿射算术，以计算DNN的绝对误差和相对误差界。我们用几个示例演示了我们的工具。
Deep Neural Networks (DNN) represent a performance-hungry application. Floating-Point (FP) and custom floating-point-like arithmetic satisfies this hunger. While there is need for speed, inference in DNNs does not seem to have any need for precision. Many papers experimentally observe that DNNs can successfully run at almost ridiculously low precision. The aim of this paper is two-fold: first, to shed some theoretical light upon why a DNN's FP accuracy stays high for low FP precision. We observe that the loss of relative accuracy in the convolutional steps is recovered by the activation layers, which are extremely well-conditioned. We give an interpretation for the link between precision and accuracy in DNNs. Second, the paper presents a software framework for semi-automatic FP error analysis for the inference phase of deep-learning. Compatible with common Tensorflow/Keras models, it leverages the frugally-deep Python/C++ library to transform a neural network into C++ code in order to analyze the network's need for precision. This rigorous analysis is based on Interval and Affine arithmetics to compute absolute and relative error bounds for a DNN. We demonstrate our tool with several examples.
