论文标题
用于峰值神经网络的时间替代后传播
Temporal Surrogate Back-propagation for Spiking Neural Networks
论文作者
论文摘要
与人工神经网络（ANN）相比，尖峰神经网络（SNN）通常更节能，并且它们的工作方式与我们的大脑具有很大的相似性。近年来，后传播（BP）表明了其在训练ANN方面的强大力量。但是，由于峰值行为是不可差异的，因此BP不能直接应用于SNN。尽管先前的工作证明了通过替代梯度或随机性在空间和时间方向上近似BP级别的几种方法，但它们省略了每个步骤之间的重置机制引入的时间依赖性。在本文中，我们针对理论完成并彻底研究缺失术语的效果。通过添加重置机制的时间依赖性，新算法对于在玩具数据集上的学习率调整更为强大，但在CIFAR-10（例如CIFAR-10）上没有太大改进。从经验上讲，缺少术语的好处不值得额外的计算开销。在许多情况下，丢失的术语可以忽略。
Spiking neural networks (SNN) are usually more energy-efficient as compared to Artificial neural networks (ANN), and the way they work has a great similarity with our brain. Back-propagation (BP) has shown its strong power in training ANN in recent years. However, since spike behavior is non-differentiable, BP cannot be applied to SNN directly. Although prior works demonstrated several ways to approximate the BP-gradient in both spatial and temporal directions either through surrogate gradient or randomness, they omitted the temporal dependency introduced by the reset mechanism between each step. In this article, we target on theoretical completion and investigate the effect of the missing term thoroughly. By adding the temporal dependency of the reset mechanism, the new algorithm is more robust to learning-rate adjustments on a toy dataset but does not show much improvement on larger learning tasks like CIFAR-10. Empirically speaking, the benefits of the missing term are not worth the additional computational overhead. In many cases, the missing term can be ignored.
