论文标题
通过Gumbel-Max方程学习符号表达式学习者网络
Learning Symbolic Expressions via Gumbel-Max Equation Learner Networks
论文作者
论文摘要
通过最先进的机器学习技术学到的大多数神经网络（NNS）都是黑盒模型。为了在科学和工程学中取得广泛的成功，重要的是开发新的NN体系结构以有效地从复杂的数据集中提取高级数学知识。在这种理解的推动下，本文开发了一种新的NN体系结构，称为Gumbel-Max方程学习者（GMEQL）网络。与先前提出的方程学习者（EQL）网络不同，GMEQL通过Gumbel-Max Trick将连续放松应用于网络结构，并引入了两种类型的可训练参数：结构参数和回归参数。本文还提出了一个两阶段的培训过程，采用新技术，以基于精英存储库的在线和离线设置进行培训结构参数。在8个基准测试符号回归问题上，GMEQL在实验上显示出优于几种尖端的机器学习方法。
Most of the neural networks (NNs) learned via state-of-the-art machine learning techniques are black-box models. For a widespread success of machine learning in science and engineering, it is important to develop new NN architectures to effectively extract high-level mathematical knowledge from complex datasets. Motivated by this understanding, this paper develops a new NN architecture called the Gumbel-Max Equation Learner (GMEQL) network. Different from previously proposed Equation Learner (EQL) networks, GMEQL applies continuous relaxation to the network structure via the Gumbel-Max trick and introduces two types of trainable parameters: structure parameters and regression parameters. This paper also proposes a two-stage training process with new techniques to train structure parameters in both online and offline settings based on an elite repository. On 8 benchmark symbolic regression problems, GMEQL is experimentally shown to outperform several cutting-edge machine learning approaches.
