论文标题
扩展室外范围：在多个子域中检查质量检查模型
Extending the Scope of Out-of-Domain: Examining QA models in multiple subdomains
论文作者
论文摘要
研究质量检查系统室外性能的过去工作主要集中在通用域（例如新闻领域，Wikipedia域），低估了由QA数据集的内部特征定义的子域的重要性。在本文中，我们根据其几个内部特征，包括问题类型，文本长度，答案位置，将质量检查示例分为不同的子域，来扩展“室外”的范围。然后，我们检查了经过不同子域数据训练的质量检查系统的性能。实验结果表明，当火车数据和测试数据来自不同的子域时，质量检查系统的性能可以大大降低。这些结果质疑当前质量检查系统在多个子域中的普遍性，这表明需要对抗质量检查数据集的内部特征引入的偏差。
Past works that investigate out-of-domain performance of QA systems have mainly focused on general domains (e.g. news domain, wikipedia domain), underestimating the importance of subdomains defined by the internal characteristics of QA datasets. In this paper, we extend the scope of "out-of-domain" by splitting QA examples into different subdomains according to their several internal characteristics including question type, text length, answer position. We then examine the performance of QA systems trained on the data from different subdomains. Experimental results show that the performance of QA systems can be significantly reduced when the train data and test data come from different subdomains. These results question the generalizability of current QA systems in multiple subdomains, suggesting the need to combat the bias introduced by the internal characteristics of QA datasets.
