论文标题
阈值线性网络的组合几何形状
Combinatorial Geometry of Threshold-Linear Networks
论文作者
论文摘要
神经网络的体系结构限制了可能出现的潜在动态。有些体系结构可能只允许单个动态制度,而另一些架构则显示出很大的灵活性,可以通过调节连接强度来实现定性不同的动态。在这项工作中,我们开发了新颖的数学技术来研究在竞争性阈值线性网络(TLNS)背景下,不同网络体系结构施加的动态约束。任何给定的TLN都自然地以$ \ Mathbb {r}^n $中的超平面布置为特征,并且该布置的组合属性确定了动力学的固定点的模式。该观察结果使我们能够在定向的矩形语言中重新提出网络灵活性问题,从而使我们能够采用该理论的工具和结果,以表征给定体系结构可以支持的不同动态制度。特别是,TLN的固定点对应于相关的矩阵的cocircit。和矩形的突变对应于固定点收集中的分叉。作为一个应用程序,我们提供了所有可能在网络中通过尺寸$ n = 3 $的固定点组的完整表征,以及如何调节网络的突触强度的描述,以访问不同的动态制度。这些结果为研究在实际神经网络中观察到的各种基序的可能计算作用提供了一个框架。
The architecture of a neural network constrains the potential dynamics that can emerge. Some architectures may only allow for a single dynamic regime, while others display a great deal of flexibility with qualitatively different dynamics that can be reached by modulating connection strengths. In this work, we develop novel mathematical techniques to study the dynamic constraints imposed by different network architectures in the context of competitive threshold-linear networks (TLNs). Any given TLN is naturally characterized by a hyperplane arrangement in $\mathbb{R}^n$, and the combinatorial properties of this arrangement determine the pattern of fixed points of the dynamics. This observation enables us to recast the question of network flexibility in the language of oriented matroids, allowing us to employ tools and results from this theory in order to characterize the different dynamic regimes a given architecture can support. In particular, fixed points of a TLN correspond to cocircuits of an associated oriented matroid; and mutations of the matroid correspond to bifurcations in the collection of fixed points. As an application, we provide a complete characterization of all possible sets of fixed points that can arise in networks through size $n=3$, together with descriptions of how to modulate synaptic strengths of the network in order to access the different dynamic regimes. These results provide a framework for studying the possible computational roles of various motifs observed in real neural networks.