论文标题
多语言审计语言模型的事实一致性
Factual Consistency of Multilingual Pretrained Language Models
论文作者
论文摘要
可以在知识知识的情况下查询验证的语言模型，并在知识基础获取和需要推理的任务中使用潜在的应用。但是，为此，我们需要知道这一知识的可靠性，最近的工作表明，单语言英语模型在预测事实知识时缺乏一致性，即它们在描述相同事实的释义方面以不同的方式填充。在本文中，我们将一致性的分析扩展到多语言设置。我们介绍了一种资源，mpararel和（i）（i）诸如Mbert和XLM-R之类的多语言模型是否比单语言对应物更一致； （ii）如果此类模型在各种语言之间同样一致。我们发现，姆伯特在英语释义中和英语伯特一样不一致，但是姆伯特和xlm-r在英语方面都表现出很高的不一致性，而对于所有其他45种语言来说，莫伯特和XLM-R都具有很高的不一致性。
Pretrained language models can be queried for factual knowledge, with potential applications in knowledge base acquisition and tasks that require inference. However, for that, we need to know how reliable this knowledge is, and recent work has shown that monolingual English language models lack consistency when predicting factual knowledge, that is, they fill-in-the-blank differently for paraphrases describing the same fact. In this paper, we extend the analysis of consistency to a multilingual setting. We introduce a resource, mParaRel, and investigate (i) whether multilingual language models such as mBERT and XLM-R are more consistent than their monolingual counterparts; and (ii) if such models are equally consistent across languages. We find that mBERT is as inconsistent as English BERT in English paraphrases, but that both mBERT and XLM-R exhibit a high degree of inconsistency in English and even more so for all the other 45 languages.
