论文标题
使用分形维度分析回波状态网络
Analyzing Echo-state Networks Using Fractal Dimension
论文作者
论文摘要
这项工作与储层优化，信息理论最佳编码以及中心分形分析的各个方面相结合。我们基于这样的观察，即由于复发性神经网络的递归性质，输入序列在其隐藏状态表示中以分形模式出现。这些模式的分形尺寸低于储层中的单元数量。我们在优化复发性神经网络初始化方面显示了这种分形维度的潜在用法。我们将“理想”储层的想法与使用算术编码器的无损最佳编码联系起来。我们的调查表明，从输入到隐藏状态的映射的分形维度应接近网络中的单位数量。分形维度和网络连接性之间的这种连接是复发神经网络初始化和储层计算的有趣的新方向。
This work joins aspects of reservoir optimization, information-theoretic optimal encoding, and at its center fractal analysis. We build on the observation that, due to the recursive nature of recurrent neural networks, input sequences appear as fractal patterns in their hidden state representation. These patterns have a fractal dimension that is lower than the number of units in the reservoir. We show potential usage of this fractal dimension with regard to optimization of recurrent neural network initialization. We connect the idea of `ideal' reservoirs to lossless optimal encoding using arithmetic encoders. Our investigation suggests that the fractal dimension of the mapping from input to hidden state shall be close to the number of units in the network. This connection between fractal dimension and network connectivity is an interesting new direction for recurrent neural network initialization and reservoir computing.
