论文标题
掩盖语言建模和适配器的有效性，用于事实知识注入
The Effectiveness of Masked Language Modeling and Adapters for Factual Knowledge Injection
论文作者
论文摘要
本文研究了将事实知识注入大型预训练的语言模型的问题。我们使用掩盖语言建模目标在概念网知识图的一部分上训练适配器模块，并通过对喇嘛探测器上的一系列探测实验来评估该方法的成功。用于不同配置的平均p@k曲线表明该技术是有效的，通过在原始模型中添加高达2.1％的附加参数，从而提高了LAMA探针的子集的性能。
This paper studies the problem of injecting factual knowledge into large pre-trained language models. We train adapter modules on parts of the ConceptNet knowledge graph using the masked language modeling objective and evaluate the success of the method by a series of probing experiments on the LAMA probe. Mean P@K curves for different configurations indicate that the technique is effective, increasing the performance on subsets of the LAMA probe for large values of k by adding as little as 2.1% additional parameters to the original models.
