论文标题
探索人工神经网络中的明确粗粒结构
Exploring explicit coarse-grained structure in artificial neural networks
论文作者
论文摘要
我们建议在人工神经网络中采用分层粗粒结构，以明确提高可解释性而不会降低性能。这个想法已在两种情况下应用。一个是一个称为Taylornet的神经网络，其目的是直接在Taylor系列方面近似从输入数据到输出结果的一般映射，而无需诉诸任何魔术非线性激活。另一个是用于数据蒸馏的新设置，可以执行输入数据集的多级抽象，并生成具有原始数据集相关功能的新数据，并可以用作分类的参考。在这两种情况下，粗粒结构在简化网络并提高解释性和效率方面都起着重要作用。在MNIST和CIFAR-10数据集上已证明了该有效性。还讨论了进一步的改进和一些与之相关的开放问题。
We propose to employ the hierarchical coarse-grained structure in the artificial neural networks explicitly to improve the interpretability without degrading performance. The idea has been applied in two situations. One is a neural network called TaylorNet, which aims to approximate the general mapping from input data to output result in terms of Taylor series directly, without resorting to any magic nonlinear activations. The other is a new setup for data distillation, which can perform multi-level abstraction of the input dataset and generate new data that possesses the relevant features of the original dataset and can be used as references for classification. In both cases, the coarse-grained structure plays an important role in simplifying the network and improving both the interpretability and efficiency. The validity has been demonstrated on MNIST and CIFAR-10 datasets. Further improvement and some open questions related are also discussed.
