论文标题
有效的对话建模的状态内存调节变压器
Stateful Memory-Augmented Transformers for Efficient Dialogue Modeling
论文作者
论文摘要
变形金刚编码器模型在对话生成任务中取得了出色的性能，但是，它们无法处理长时间的对话历史记录通常会导致上下文解决此问题，我们提出了一种新型的内存效果型变压器，该变形金刚与现有的预先培训的编码器模型兼容，并能够对对话历史记录有效地保存对话历史信息。通过将单独的内存模块与预训练的变压器旁边合并，模型可以有效地在内存状态和当前输入上下文之间进行互换信息。我们在三个对话数据集和两个语言建模数据集上评估了我们的模型。实验结果表明，与其他预训练的变压器基线相比，我们的方法已经达到了较高的效率和性能。
Transformer encoder-decoder models have achieved great performance in dialogue generation tasks, however, their inability to process long dialogue history often leads to truncation of the context To address this problem, we propose a novel memory-augmented transformer that is compatible with existing pre-trained encoder-decoder models and enables efficient preservation of the dialogue history information. By incorporating a separate memory module alongside the pre-trained transformer, the model can effectively interchange information between the memory states and the current input context. We evaluate our model on three dialogue datasets and two language modeling datasets. Experimental results show that our method has achieved superior efficiency and performance compared to other pre-trained Transformer baselines.
