论文标题
RUATD共享任务2022关于俄罗斯人的人造文本检测的结果
Findings of the The RuATD Shared Task 2022 on Artificial Text Detection in Russian
论文作者
论文摘要
我们介绍了俄罗斯人的人工文本检测的共同任务，该任务是2022年举行的对话评估计划的一部分。共享的任务数据集包括来自14个文本生成器的文本，即一位人类作家和13个文本生成模型，用于以下一代或多个一代或多个任务：机器翻译，拼写，拼写，文本简化，文本简化。我们还考虑了反向翻译和零发的生成方法。人写的文本是从多个领域的公开资源中收集的。共享任务由两个子任务组成：（i）确定给定文本是由人类自动生成或编写的； （ii）确定给定文本的作者。第一个任务是将其构架为二进制分类问题。第二个任务是多类分类问题。我们提供基于计数和基于BERT的基线，以及对第一个子任务的人类评估。相应地，总共向二进制和多级子任务提交了30和8个系统。大多数球队的表现都超过了基线。我们在GitHub存储库中公开发布代码库，人类评估结果和其他材料（https://github.com/dialogue-evaluation/ruatd）。
We present the shared task on artificial text detection in Russian, which is organized as a part of the Dialogue Evaluation initiative, held in 2022. The shared task dataset includes texts from 14 text generators, i.e., one human writer and 13 text generative models fine-tuned for one or more of the following generation tasks: machine translation, paraphrase generation, text summarization, text simplification. We also consider back-translation and zero-shot generation approaches. The human-written texts are collected from publicly available resources across multiple domains. The shared task consists of two sub-tasks: (i) to determine if a given text is automatically generated or written by a human; (ii) to identify the author of a given text. The first task is framed as a binary classification problem. The second task is a multi-class classification problem. We provide count-based and BERT-based baselines, along with the human evaluation on the first sub-task. A total of 30 and 8 systems have been submitted to the binary and multi-class sub-tasks, correspondingly. Most teams outperform the baselines by a wide margin. We publicly release our codebase, human evaluation results, and other materials in our GitHub repository (https://github.com/dialogue-evaluation/RuATD).
