论文标题
卷积重量分布假设：重新思考频道修剪的标准
Convolution-Weight-Distribution Assumption: Rethinking the Criteria of Channel Pruning
论文作者
论文摘要
通道修剪是压缩卷积神经网络（CNN）的一种流行技术，已提出各种修剪标准来删除冗余过滤器。从我们的全面实验中，我们发现了修剪标准的研究中有两个盲点：（1）相似性：几个主要的修剪标准中有一些强烈的相似性，这些标准被广泛引用和比较。根据这些标准，滤波器的物质得分的等级几乎相同，从而产生了相似的修剪结构。 （2）适用性：通过某些修剪标准测量的过滤器物质得分太近，无法很好地区分网络冗余。在本文中，我们在不同类型的修剪标准上分析了这两个盲点，并通过层修剪或全球修剪。这些分析基于经验实验和我们的假设（卷积重量分布假设），即受过训练的卷积过滤每个层大约遵循高斯类似分布。通过系统和广泛的统计检验验证了这一假设。
Channel pruning is a popular technique for compressing convolutional neural networks (CNNs), where various pruning criteria have been proposed to remove the redundant filters. From our comprehensive experiments, we found two blind spots in the study of pruning criteria: (1) Similarity: There are some strong similarities among several primary pruning criteria that are widely cited and compared. According to these criteria, the ranks of filters'Importance Score are almost identical, resulting in similar pruned structures. (2) Applicability: The filters'Importance Score measured by some pruning criteria are too close to distinguish the network redundancy well. In this paper, we analyze these two blind spots on different types of pruning criteria with layer-wise pruning or global pruning. The analyses are based on the empirical experiments and our assumption (Convolutional Weight Distribution Assumption) that the well-trained convolutional filters each layer approximately follow a Gaussian-alike distribution. This assumption has been verified through systematic and extensive statistical tests.
