论文标题
可证明稳定的神经网络图灵机
A provably stable neural network Turing Machine
论文作者
论文摘要
我们介绍了一个神经堆栈体系结构，包括一个可区分的参数化堆栈操作员，该堆栈操作员近似堆栈推动和弹出操作，以选择明确表示堆栈的参数选择。我们证明了这种堆栈体系结构的稳定性：在任意许多堆栈操作之后，神经堆栈的状态仍然与离散堆栈的状态非常相似。使用神经堆栈和复发性神经网络，我们引入了神经网络下降自动机（NNPDA），并证明具有有限/有界神经元的NNPDA可以模拟任何PDA。此外，我们扩展了建筑，并提出了新的建筑神经状态图灵机（NNTM）。我们证明，具有有界神经元的可区分NNTM可以实时模拟图灵机（TM）。就像神经堆栈一样，这些架构也很稳定。最后，我们扩展了构造，以表明可区分的NNTM等同于通用图灵机（UTM），并且只能使用\ textbf {七个有限/有限的精度}神经元模拟任何TM。这项工作为有限的精度RNN的计算能力提供了新的理论界限，并随着内存增强。
We introduce a neural stack architecture, including a differentiable parametrized stack operator that approximates stack push and pop operations for suitable choices of parameters that explicitly represents a stack. We prove the stability of this stack architecture: after arbitrarily many stack operations, the state of the neural stack still closely resembles the state of the discrete stack. Using the neural stack with a recurrent neural network, we introduce a neural network Pushdown Automaton (nnPDA) and prove that nnPDA with finite/bounded neurons and time can simulate any PDA. Furthermore, we extend our construction and propose new architecture neural state Turing Machine (nnTM). We prove that differentiable nnTM with bounded neurons can simulate Turing Machine (TM) in real-time. Just like the neural stack, these architectures are also stable. Finally, we extend our construction to show that differentiable nnTM is equivalent to Universal Turing Machine (UTM) and can simulate any TM with only \textbf{seven finite/bounded precision} neurons. This work provides a new theoretical bound for the computational capability of bounded precision RNNs augmented with memory.
