论文标题
可控的自然语言产生具有对比前缀
Controllable Natural Language Generation with Contrastive Prefixes
论文作者
论文摘要
为了指导大型语言模型（LM）的产生，以前的工作集中于直接微调语言模型或使用属性歧视器。在这项工作中，我们为可控的GPT2一代提出了一个新颖的轻量级框架，该框架利用一组小属性特定的向量（称为前缀）来引导自然语言生成。与前缀调整不同，在每个前缀都经过独立训练的情况下，我们考虑了前缀之间的关系，并同时训练多个前缀。我们提出了一种新颖的监督方法，也提出了一种无监督的方法，可以训练前缀进行单光检查控制，而这两种方法的组合可以实现多主体控制。单位和多光值控制的实验结果表明，我们的方法可以在保持高语言质量的同时引导产生所需的属性。
To guide the generation of large pretrained language models (LM), previous work has focused on directly fine-tuning the language model or utilizing an attribute discriminator. In this work, we propose a novel lightweight framework for controllable GPT2 generation, which utilizes a set of small attribute-specific vectors, called prefixes, to steer natural language generation. Different from prefix-tuning, where each prefix is trained independently, we take the relationship among prefixes into consideration and train multiple prefixes simultaneously. We propose a novel supervised method and also an unsupervised method to train the prefixes for single-aspect control while the combination of these two methods can achieve multi-aspect control. Experimental results on both single-aspect and multi-aspect control show that our methods can guide generation towards the desired attributes while keeping high linguistic quality.
