论文标题
量化网络中的批准化
Batch Normalization in Quantized Networks
论文作者
论文摘要
在计算硬件上实施量化的神经网络会导致相当大的速度和保存记忆。但是，量化的深网很难训练和批处理标准化（BatchNorm）层在训练完整精确和量化网络中起重要作用。大多数关于BATCHNORM的研究都集中在完整的网络上，并且在了解量化培训中的batchNorm影响方面几乎没有研究。我们表明，批评是避免了违反直觉的梯度爆炸，并且最近在数值实验中观察到了其他研究人员。
Implementation of quantized neural networks on computing hardware leads to considerable speed up and memory saving. However, quantized deep networks are difficult to train and batch~normalization (BatchNorm) layer plays an important role in training full-precision and quantized networks. Most studies on BatchNorm are focused on full-precision networks, and there is little research in understanding BatchNorm affect in quantized training which we address here. We show BatchNorm avoids gradient explosion which is counter-intuitive and recently observed in numerical experiments by other researchers.
