论文标题
薄荷：通过基于信息的神经元修剪的深层网络压缩
MINT: Deep Network Compression via Mutual Information-based Neuron Trimming
论文作者
论文摘要
大多数通过修剪通过修剪来评估过滤器的权重，或者使用稀疏性约束来评估过滤器的重要性或优化替代目标函数的大多数方法。尽管这些方法为近似类似过滤器的贡献提供了一种有用的方法，但它们通常会忽略层之间的依赖性，或者要比标准的跨透镜更难求解更难的优化目标。我们的方法，基于信息的神经元修剪（MINT），通过根据每对层之间的相邻层过滤器之间的关系强度来实现稀疏性，通过修剪来进行深度压缩。关系是使用条件几何互信息来计算的，该信息评估了使用基于图的标准在过滤器之间交换的相似信息的数量。修剪网络时，我们确保保留过滤器将大部分信息贡献出可确保高性能的后续层。我们的新方法在标准基准上的现有最新的压缩 - 预见方法优于此任务的标准基准：MNIST，CIFAR-10和ILSVRC2012在各种网络架构上。此外，与原始网络相比，我们讨论了我们修剪方法对对抗攻击的反应和校准统计数据之间对共同点的观察结果。
Most approaches to deep neural network compression via pruning either evaluate a filter's importance using its weights or optimize an alternative objective function with sparsity constraints. While these methods offer a useful way to approximate contributions from similar filters, they often either ignore the dependency between layers or solve a more difficult optimization objective than standard cross-entropy. Our method, Mutual Information-based Neuron Trimming (MINT), approaches deep compression via pruning by enforcing sparsity based on the strength of the relationship between filters of adjacent layers, across every pair of layers. The relationship is calculated using conditional geometric mutual information which evaluates the amount of similar information exchanged between the filters using a graph-based criterion. When pruning a network, we ensure that retained filters contribute the majority of the information towards succeeding layers which ensures high performance. Our novel approach outperforms existing state-of-the-art compression-via-pruning methods on the standard benchmarks for this task: MNIST, CIFAR-10, and ILSVRC2012, across a variety of network architectures. In addition, we discuss our observations of a common denominator between our pruning methodology's response to adversarial attacks and calibration statistics when compared to the original network.
