论文标题
参考和文档意识的韩语语言评估方法摘要
Reference and Document Aware Semantic Evaluation Methods for Korean Language Summarization
论文作者
论文摘要
文本摘要是指从源文档中生成更短的文本形式的过程。通常通过使用面向召回的研究来评估（Rouge）分数来评估许多现有的文本摘要作品。但是，由于基于n-gram重叠计算胭脂分数，它们并不能反映生成和参考摘要之间的语义含义对应关系。由于韩语是一种将各种词素结合到表达几种含义的单词中的凝聚力语言，因此Rouge不适合韩国摘要。在本文中，我们提出的评估指标反映了参考摘要的语义含义以及原始文档，参考和文档意识语义分数（RDASS）。然后，我们提出了一种改善指标与人类判断的相关性的方法。评估结果表明，我们的评估指标与人类判断的相关性明显高于胭脂分数。
Text summarization refers to the process that generates a shorter form of text from the source document preserving salient information. Many existing works for text summarization are generally evaluated by using recall-oriented understudy for gisting evaluation (ROUGE) scores. However, as ROUGE scores are computed based on n-gram overlap, they do not reflect semantic meaning correspondences between generated and reference summaries. Because Korean is an agglutinative language that combines various morphemes into a word that express several meanings, ROUGE is not suitable for Korean summarization. In this paper, we propose evaluation metrics that reflect semantic meanings of a reference summary and the original document, Reference and Document Aware Semantic Score (RDASS). We then propose a method for improving the correlation of the metrics with human judgment. Evaluation results show that the correlation with human judgment is significantly higher for our evaluation metrics than for ROUGE scores.
