论文标题
上下文化时间图的神经语言建模
Neural Language Modeling for Contextualized Temporal Graph Generation
论文作者
论文摘要
本文介绍了第一项有关使用大规模预训练的语言模型来自动生成文档的时间级时间图的研究。尽管神经预训练方法在NLP任务中取得了巨大的成功，但其在事件图上的时间推理的潜力尚未得到充分探索。部分原因是难以通过人类宣传的事件和时间联系获得大型培训语料库。我们通过使用现有的IE/NLP工具自动生成大量（89,000）系统生产的文档编写对，并提出了将上下文化的图形生成问题作为序列到序列映射任务的新颖表述。这些策略使我们能够在系统诱导的培训数据中利用和微调预训练的语言模型。我们的实验表明，我们的方法在生成结构和语义有效的图表方面非常有效。此外，对具有挑战性的手工标记的外域语料库的评估表明，我们的方法在几个指标上的大幅度优于现有方法。代码和预训练模型可在https://github.com/madaan/temporal-graph-gen上找到。
This paper presents the first study on using large-scale pre-trained language models for automated generation of an event-level temporal graph for a document. Despite the huge success of neural pre-training methods in NLP tasks, its potential for temporal reasoning over event graphs has not been sufficiently explored. Part of the reason is the difficulty in obtaining large training corpora with human-annotated events and temporal links. We address this challenge by using existing IE/NLP tools to automatically generate a large quantity (89,000) of system-produced document-graph pairs, and propose a novel formulation of the contextualized graph generation problem as a sequence-to-sequence mapping task. These strategies enable us to leverage and fine-tune pre-trained language models on the system-induced training data for the graph generation task. Our experiments show that our approach is highly effective in generating structurally and semantically valid graphs. Further, evaluation on a challenging hand-labeled, out-domain corpus shows that our method outperforms the closest existing method by a large margin on several metrics. Code and pre-trained models are available at https://github.com/madaan/temporal-graph-gen.
