论文标题
水库的复杂拓扑特征在受生物启发的复发神经网络中形成学习表现
Complex topological features of reservoirs shape learning performances in bio-inspired recurrent neural networks
论文作者
论文摘要
经常性网络是一种特殊的人工神经系统,它们使用其内部状态来执行机器学习的计算任务。其最先进的开发之一,即储层计算(RC),它使用内部结构(通常是具有随机结构的静态网络)将输入信号映射到在较高维空间中定义的非线性动态系统中。储层的特征是其单位之间的非线性相互作用以及它们通过经常循环存储信息的能力,从而训练人造系统学习特定于任务的动态。但是,从根本上说,储层的随机拓扑如何影响学习绩效。在这里,我们通过考虑一系列的合成网络(以不同的拓扑特征为特征)和45个经验连接组来填补这一空白,并从属于8种不同物种的生物的大脑区域进行采样,以建立储层,并通过各种复杂的输入信号来测试学习绩效。我们发现RC性能与激活状态的协方差矩阵的数量和等级之间的非平凡相关性,其性能取决于输入信号的性质 - 随机或确定性 - 的性质。值得注意的是,发现储层的模块化和链路密度会影响RC的性能:这些结果无法通过模型仅考虑储层的简单拓扑特征来预测。总体而言,我们的发现凸显了表征生物物理计算系统(例如连接组)的复杂拓扑特征,可用于设计有效的生物启发的人工神经网络。
Recurrent networks are a special class of artificial neural systems that use their internal states to perform computing tasks for machine learning. One of its state-of-the-art developments, i.e. reservoir computing (RC), uses the internal structure -- usually a static network with random structure -- to map an input signal into a nonlinear dynamical system defined in a higher dimensional space. Reservoirs are characterized by nonlinear interactions among their units and their ability to store information through recurrent loops, allowing to train artificial systems to learn task-specific dynamics. However, it is fundamentally unknown how the random topology of the reservoir affects the learning performance. Here, we fill this gap by considering a battery of synthetic networks -- characterized by different topological features -- and 45 empirical connectomes -- sampled from brain regions of organisms belonging to 8 different species -- to build the reservoir and testing the learning performance against a prediction task with a variety of complex input signals. We find nontrivial correlations between RC performances and both the number of nodes and rank of the covariance matrix of activation states, with performance depending on the nature -- stochastic or deterministic -- of input signals. Remarkably, the modularity and the link density of the reservoir are found to affect RC performances: these results cannot be predicted by models only accounting for simple topological features of the reservoir. Overall, our findings highlight that the complex topological features characterizing biophysical computing systems such as connectomes can be used to design efficient bio-inspired artificial neural networks.