论文标题
几乎没有图案探索培训的文字生成
Few-Shot Text Generation with Pattern-Exploiting Training
论文作者
论文摘要
用自然语言提供简单的任务描述，使他们能够完全无监督的方式解决某些任务。此外，当与示例中的定期学习结合使用时，对于广泛的文本分类任务，这一想法会产生令人印象深刻的几次结果。这也是提高生成设置数据效率的有前途的方向，但是要组合任务描述和基于示例的学习来生成文本生成，存在一些挑战。特别是，找到易于理解的验证模型的任务描述至关重要，并确保它实际上可以充分利用它们。此外，必须实施有效的反对过度拟合的措施。在本文中，我们展示了如何解决这些挑战：我们介绍了GenPet，这是一种基于模式开发培训的文本生成方法，这是一种将文本指令与仅适用于分类任务有效的监督学习相结合的方法。在几个摘要和标题生成数据集中，GenPet在几个弹药设置中对强基础进行了一致的改进。
Providing pretrained language models with simple task descriptions in natural language enables them to solve some tasks in a fully unsupervised fashion. Moreover, when combined with regular learning from examples, this idea yields impressive few-shot results for a wide range of text classification tasks. It is also a promising direction to improve data efficiency in generative settings, but there are several challenges to using a combination of task descriptions and example-based learning for text generation. In particular, it is crucial to find task descriptions that are easy to understand for the pretrained model and to ensure that it actually makes good use of them; furthermore, effective measures against overfitting have to be implemented. In this paper, we show how these challenges can be tackled: We introduce GenPET, a method for text generation that is based on pattern-exploiting training, a recent approach for combining textual instructions with supervised learning that only works for classification tasks. On several summarization and headline generation datasets, GenPET gives consistent improvements over strong baselines in few-shot settings.
