论文标题
多模式的类比推理在知识图上
Multimodal Analogical Reasoning over Knowledge Graphs
论文作者
论文摘要
类比推理是人类认知的基础，在各个领域都有重要的位置。但是，先前的研究主要集中于单模式类似推理，而忽略利用结构知识的优势。值得注意的是，认知心理学的研究表明，来自多模式来源的信息总是比单一模态来源具有更强大的认知转移。为此，我们在知识图上介绍了多模式类似推理的新任务，这需要借助背景知识的多模式推理能力。具体而言，我们构建了一个多模式类似推理数据集（MARS）和多模式知识图标记。我们使用多模式知识图嵌入和预训练的变压器基线进行评估，说明了所提出的任务的潜在挑战。我们进一步提出了一个新型模型无形的多模式类似推理框架，该框架是由结构映射理论动机的变压器（MART），可以获得更好的性能。代码和数据集可在https://github.com/zjunlp/mkg_analogy中找到。
Analogical reasoning is fundamental to human cognition and holds an important place in various fields. However, previous studies mainly focus on single-modal analogical reasoning and ignore taking advantage of structure knowledge. Notably, the research in cognitive psychology has demonstrated that information from multimodal sources always brings more powerful cognitive transfer than single modality sources. To this end, we introduce the new task of multimodal analogical reasoning over knowledge graphs, which requires multimodal reasoning ability with the help of background knowledge. Specifically, we construct a Multimodal Analogical Reasoning dataSet (MARS) and a multimodal knowledge graph MarKG. We evaluate with multimodal knowledge graph embedding and pre-trained Transformer baselines, illustrating the potential challenges of the proposed task. We further propose a novel model-agnostic Multimodal analogical reasoning framework with Transformer (MarT) motivated by the structure mapping theory, which can obtain better performance. Code and datasets are available in https://github.com/zjunlp/MKG_Analogy.
