论文标题
调查自然语言推断分歧的原因
Investigating Reasons for Disagreement in Natural Language Inference
论文作者
论文摘要
我们研究了自然语言推论（NLI）注释的分歧。我们开发了一种分类来源的分类法，其中10个类别涵盖了3个高级类别。我们发现，某些分歧是由于句子含义的不确定性所致，而另一些分歧是对注释偏见和任务工件的，从而导致对标签分布的不同解释。我们探索了两种用于检测具有潜在分歧的项目的建模方法：除了三个标准NLI标签外，具有“复杂”标签的四向分类以及一种多标签分类方法。我们发现，多标签分类更具表现力，并更好地回忆了数据中可能的解释。
We investigate how disagreement in natural language inference (NLI) annotation arises. We developed a taxonomy of disagreement sources with 10 categories spanning 3 high-level classes. We found that some disagreements are due to uncertainty in the sentence meaning, others to annotator biases and task artifacts, leading to different interpretations of the label distribution. We explore two modeling approaches for detecting items with potential disagreement: a 4-way classification with a "Complicated" label in addition to the three standard NLI labels, and a multilabel classification approach. We found that the multilabel classification is more expressive and gives better recall of the possible interpretations in the data.
