论文标题
GAAF：通过遗传算法搜索激活功能以搜索二进制神经网络
GAAF: Searching Activation Functions for Binary Neural Networks through Genetic Algorithm
论文作者
论文摘要
二进制神经网络（BNN）显示出在成本和功率限制域（例如边缘设备和移动系统）的有希望的利用。这是由于其计算和存储需求明显较少，但其成本降低了性能。为了缩小准确性差距，在本文中，我们建议在基于符号的二进制之前添加补充激活函数（AF），并依靠遗传算法（GA）自动搜索理想的AFS。这些AFS可以帮助从向前通过的输入数据中提取额外的信息，同时允许向后通过的梯度近似。通过我们的基于GA的搜索确定了15个新颖的AFS，而在不同的数据集和网络模型上测试时，大多数AFS的性能提高了（Imagenet上的2.54％）。我们的方法为设计一般和应用特定的BNN体系结构提供了一种新颖的方法。我们的代码可在http://github.com/flying-yan/gaaf上找到。
Binary neural networks (BNNs) show promising utilization in cost and power-restricted domains such as edge devices and mobile systems. This is due to its significantly less computation and storage demand, but at the cost of degraded performance. To close the accuracy gap, in this paper we propose to add a complementary activation function (AF) ahead of the sign based binarization, and rely on the genetic algorithm (GA) to automatically search for the ideal AFs. These AFs can help extract extra information from the input data in the forward pass, while allowing improved gradient approximation in the backward pass. Fifteen novel AFs are identified through our GA-based search, while most of them show improved performance (up to 2.54% on ImageNet) when testing on different datasets and network models. Our method offers a novel approach for designing general and application-specific BNN architecture. Our code is available at http://github.com/flying-Yan/GAAF.
