论文标题
培训DNS中的归一化技术：方法，分析和应用
Normalization Techniques in Training DNNs: Methodology, Analysis and Application
论文作者
论文摘要
标准化技术对于加速训练和改善深神经网络（DNN）的概括至关重要，并成功地用于各种应用中。本文在DNN培训的背景下回顾了正常化方法的过去，现在和未来的评论。从优化的角度来看，我们提供了不同方法背后的主要动机的统一图片，并提出了一种分类法，以理解它们之间的相似性及其差异。具体而言，我们将最具代表性的激活方法的管道分解为三个组成部分：归一化区域分配，归一化操作和归一化表示恢复。在此过程中，我们为设计新的规范化技术提供了见解。最后，我们讨论了当前理解归一化方法的进展，并对标准化对特定任务的应用进行了全面审查，在这些任务中，它可以有效地解决关键问题。
Normalization techniques are essential for accelerating the training and improving the generalization of deep neural networks (DNNs), and have successfully been used in various applications. This paper reviews and comments on the past, present and future of normalization methods in the context of DNN training. We provide a unified picture of the main motivation behind different approaches from the perspective of optimization, and present a taxonomy for understanding the similarities and differences between them. Specifically, we decompose the pipeline of the most representative normalizing activation methods into three components: the normalization area partitioning, normalization operation and normalization representation recovery. In doing so, we provide insight for designing new normalization technique. Finally, we discuss the current progress in understanding normalization methods, and provide a comprehensive review of the applications of normalization for particular tasks, in which it can effectively solve the key issues.
