论文标题
培训数据比您想象的更有价值：通过从培训数据中检索的一种简单有效的方法
Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data
论文作者
论文摘要
通过引入外部知识，已证明基于检索的方法在NLP任务中有效。但是，大规模语料库的索引和检索带来了可观的计算成本。令人惊讶的是，我们发现从培训数据（REINA）中检索只会导致多个NLG和NLU任务的显着增长。我们检索标记的训练实例与输入文本最相似，然后将它们与输入相连以进出模型以生成输出。实验结果表明，这种简单的方法可以在各种NLU和NLG任务上实现更好的性能，包括摘要，机器翻译，语言建模和问答任务。例如，我们提出的方法在XSUM，BigPatent和CommonSenseQA上实现了最先进的结果。我们的代码发布https://github.com/microsoft/reina。
Retrieval-based methods have been shown to be effective in NLP tasks via introducing external knowledge. However, the indexing and retrieving of large-scale corpora bring considerable computational cost. Surprisingly, we found that REtrieving from the traINing datA (REINA) only can lead to significant gains on multiple NLG and NLU tasks. We retrieve the labeled training instances most similar to the input text and then concatenate them with the input to feed into the model to generate the output. Experimental results show that this simple method can achieve significantly better performance on a variety of NLU and NLG tasks, including summarization, machine translation, language modeling, and question answering tasks. For instance, our proposed method achieved state-of-the-art results on XSum, BigPatent, and CommonsenseQA. Our code is released, https://github.com/microsoft/REINA .
