论文标题
对神经机器翻译的语义表示分类
Categorizing Semantic Representations for Neural Machine Translation
论文作者
论文摘要
现代神经机器翻译（NMT）模型已在标准基准中实现了竞争性能。然而，最近已显示它们在组成概括中受到限制，无法有效地学习原子的翻译（例如单词）及其语义组成（例如修饰），从所见的化合物（例如，短语），因此在未观察到的化合物期间在杂物期间的转化性能明显弱弱。我们通过将分类引入源上下文化表示形式来解决此问题。主要思想是通过降低稀疏性和过度拟合来增强概括，这是通过在训练集中找到令牌表示的原型来实现的，并将其嵌入在源编码中。在专用MT数据集（即认知）上进行的实验表明，我们的方法通过降低24 \％的误差降低了组成概括错误率。此外，我们概念上简单的方法比在一系列MT数据集上的变压器基线提供了一致的更好结果。
Modern neural machine translation (NMT) models have achieved competitive performance in standard benchmarks. However, they have recently been shown to suffer limitation in compositional generalization, failing to effectively learn the translation of atoms (e.g., words) and their semantic composition (e.g., modification) from seen compounds (e.g., phrases), and thus suffering from significantly weakened translation performance on unseen compounds during inference. We address this issue by introducing categorization to the source contextualized representations. The main idea is to enhance generalization by reducing sparsity and overfitting, which is achieved by finding prototypes of token representations over the training set and integrating their embeddings into the source encoding. Experiments on a dedicated MT dataset (i.e., CoGnition) show that our method reduces compositional generalization error rates by 24\% error reduction. In addition, our conceptually simple method gives consistently better results than the Transformer baseline on a range of general MT datasets.
