论文标题
从不同监督信号得出的句子嵌入的比较和组合
Comparison and Combination of Sentence Embeddings Derived from Different Supervision Signals
论文作者
论文摘要
句子嵌入方法有许多成功的应用。但是，根据监督信号，在结果句子嵌入中捕获了哪些属性。在本文中，我们专注于具有相似体系结构和任务的两种句子嵌入方法：一种在自然语言推理任务上进行的微型训练性语言模型，以及其他微型训练的语言模型，从其定义句子中进行了单词预测任务，并研究其属性。具体而言，我们使用从两个角度分区的STS数据集进行了他们在语义文本相似性（STS）任务上的表现：1）句子源和2）句子对的表面相似性，并在下游和探测任务上比较其表现。此外，我们尝试结合两种方法，并证明将两种方法组合起来比无监督的STS任务和下游任务的各自方法的性能要好得多。
There have been many successful applications of sentence embedding methods. However, it has not been well understood what properties are captured in the resulting sentence embeddings depending on the supervision signals. In this paper, we focus on two types of sentence embedding methods with similar architectures and tasks: one fine-tunes pre-trained language models on the natural language inference task, and the other fine-tunes pre-trained language models on word prediction task from its definition sentence, and investigate their properties. Specifically, we compare their performances on semantic textual similarity (STS) tasks using STS datasets partitioned from two perspectives: 1) sentence source and 2) superficial similarity of the sentence pairs, and compare their performances on the downstream and probing tasks. Furthermore, we attempt to combine the two methods and demonstrate that combining the two methods yields substantially better performance than the respective methods on unsupervised STS tasks and downstream tasks.
