论文标题
一对：长期文本的预训练变压器的计划和迭代精致
PAIR: Planning and Iterative Refinement in Pre-trained Transformers for Long Text Generation
论文作者
论文摘要
预先训练的变压器在产生长长而流利的文本方面使得令人印象深刻的突破，但是它们的输出通常在没有相干布置的内容的情况下“闲逛”。在这项工作中，我们介绍了一个新颖的内容控制的文本生成框架，配对，计划和迭代精致，该框架建立在大型模型BART上。我们首先调整BERT模型以自动构建内容计划，该计划由键形分配及其相应的句子级位置组成。 BART模型用于生成而无需修改其结构。然后，我们提出了一种改进算法，以逐步提高序列到序列框架内的发电质量。使用自动指标的评估表明，添加计划一致地提高了三个不同领域的发电质量，平均有20个BLEU点和12个流星点的改善。此外，人类法官将我们的系统输出评价比没有计划的比较更相关和连贯。
Pre-trained Transformers have enabled impressive breakthroughs in generating long and fluent text, yet their outputs are often "rambling" without coherently arranged content. In this work, we present a novel content-controlled text generation framework, PAIR, with planning and iterative refinement, which is built upon a large model, BART. We first adapt the BERT model to automatically construct the content plans, consisting of keyphrase assignments and their corresponding sentence-level positions. The BART model is employed for generation without modifying its structure. We then propose a refinement algorithm to gradually enhance the generation quality within the sequence-to-sequence framework. Evaluation with automatic metrics shows that adding planning consistently improves the generation quality on three distinct domains, with an average of 20 BLEU points and 12 METEOR points improvements. In addition, human judges rate our system outputs to be more relevant and coherent than comparisons without planning.
