论文标题
低复杂性卷积神经网络的预定义稀疏性
Pre-defined Sparsity for Low-Complexity Convolutional Neural Networks
论文作者
论文摘要
处理深度卷积神经网络的高能源成本阻碍了他们在嵌入式系统和IoT设备等能源约束平台中的无处不在部署。这项工作引入了具有预定义的稀疏2D内核的卷积层，其支持集可以在过滤器内和跨过滤器内定期重复。由于我们周期性稀疏内核的有效存储，由于DRAM访问的减少，参数节省的能源效率可以大大提高，因此有望在培训和推理的能源消耗和准确性之间取舍的重大改进。为了评估这种方法，我们在RESNET18和VGG16体系结构的稀疏变体中使用了两个广泛接受的数据集CIFAR-10和TINY IMAGENET进行了实验。与基线模型相比，我们提出的稀疏变体需要少82％的模型参数，而在CIFAR-10上的RESNET18的准确性损失却忽略了5.6倍。对于接受小型成像网训练的VGG16，我们的方法需要减少5.8倍的失败，而较少的模型参数少了83.3％，TOP-5（TOP-1）精度仅下降1.2％（2.1％）。我们还将提议的体系结构的性能与Shufflenet和MobileNetV2的性能进行了比较。使用类似的超参数和拖鞋，我们的RESNET18变体的平均准确性提高了2.8％。
The high energy cost of processing deep convolutional neural networks impedes their ubiquitous deployment in energy-constrained platforms such as embedded systems and IoT devices. This work introduces convolutional layers with pre-defined sparse 2D kernels that have support sets that repeat periodically within and across filters. Due to the efficient storage of our periodic sparse kernels, the parameter savings can translate into considerable improvements in energy efficiency due to reduced DRAM accesses, thus promising significant improvements in the trade-off between energy consumption and accuracy for both training and inference. To evaluate this approach, we performed experiments with two widely accepted datasets, CIFAR-10 and Tiny ImageNet in sparse variants of the ResNet18 and VGG16 architectures. Compared to baseline models, our proposed sparse variants require up to 82% fewer model parameters with 5.6times fewer FLOPs with negligible loss in accuracy for ResNet18 on CIFAR-10. For VGG16 trained on Tiny ImageNet, our approach requires 5.8times fewer FLOPs and up to 83.3% fewer model parameters with a drop in top-5 (top-1) accuracy of only 1.2% (2.1%). We also compared the performance of our proposed architectures with that of ShuffleNet andMobileNetV2. Using similar hyperparameters and FLOPs, our ResNet18 variants yield an average accuracy improvement of 2.8%.
