论文标题
可逆神经网络的通用近似特性
Universal approximation property of invertible neural networks
论文作者
论文摘要
可逆神经网络（INNS）是具有设计可逆性的神经网络体系结构。由于它们的可逆性和雅各布的障碍性，Inns具有各种机器学习应用，例如概率建模，生成建模和代表性学习。但是，它们的吸引力通常是以限制层设计的代价，这对它们的表示能力提出了一个问题：我们可以使用这些模型来近似足够多样化的功能吗？为了回答这个问题，我们开发了一个一般的理论框架，以研究基于差异几何结构定理基础的旅馆的表示力。该框架简化了差异性的近似问题，这使我们能够显示旅馆的通用近似属性。我们将框架应用于两个代表性的旅馆类别，即基于耦合流的旅馆（CF-INNS）和神经普通微分方程（节点），并阐明了他们的高代表权，尽管限制了它们的体系结构。
Invertible neural networks (INNs) are neural network architectures with invertibility by design. Thanks to their invertibility and the tractability of Jacobian, INNs have various machine learning applications such as probabilistic modeling, generative modeling, and representation learning. However, their attractive properties often come at the cost of restricting the layer designs, which poses a question on their representation power: can we use these models to approximate sufficiently diverse functions? To answer this question, we have developed a general theoretical framework to investigate the representation power of INNs, building on a structure theorem of differential geometry. The framework simplifies the approximation problem of diffeomorphisms, which enables us to show the universal approximation properties of INNs. We apply the framework to two representative classes of INNs, namely Coupling-Flow-based INNs (CF-INNs) and Neural Ordinary Differential Equations (NODEs), and elucidate their high representation power despite the restrictions on their architectures.
