论文标题

$ \ textit {lit} $ - glat:瞥了一眼平行文本生成的潜在变量

$\textit{latent}$-GLAT: Glancing at Latent Variables for Parallel Text Generation

论文作者

Bao, Yu, Zhou, Hao, Huang, Shujian, Wang, Dongqi, Qian, Lihua, Dai, Xinyu, Chen, Jiajun, Li, Lei

论文摘要

最近,由于其在发电效率方面的成功,并行文本生成受到了广泛的关注。尽管提出了许多高级技术来提高其发电质量,但他们仍然需要自回归模型的帮助来克服数据集中的一对多多模式现象,从而限制了他们的应用程序。在本文中,我们提出了$ \ textit {lit} $ - 使用离散的潜在变量来捕获单词分类信息并调用高级课程学习技术,从而减轻了多模式问题。实验结果表明,我们的方法在没有自回归模型的帮助的情况下优于强基础,这进一步扩大了并行解码范式的应用方案。

Recently, parallel text generation has received widespread attention due to its success in generation efficiency. Although many advanced techniques are proposed to improve its generation quality, they still need the help of an autoregressive model for training to overcome the one-to-many multi-modal phenomenon in the dataset, limiting their applications. In this paper, we propose $\textit{latent}$-GLAT, which employs the discrete latent variables to capture word categorical information and invoke an advanced curriculum learning technique, alleviating the multi-modality problem. Experiment results show that our method outperforms strong baselines without the help of an autoregressive model, which further broadens the application scenarios of the parallel decoding paradigm.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源