论文标题
文本的神经嵌入
Neural Embeddings for Text
论文作者
论文摘要
我们提出了一种新型的自然语言文本嵌入，该文本深深代表语义含义。标准文本嵌入使用验证语言模型的隐藏层的输出。在我们的方法中，我们让语言模型从文本中学习，然后从字面上挑选其大脑，从而将模型神经元的实际权重产生向量。我们将此文本表示为神经嵌入。我们通过对几个数据集上的行为进行分析，以及将神经嵌入与最先进的句子嵌入的状态进行比较，证实了该表示的能力反映文本语义的能力。
We propose a new kind of embedding for natural language text that deeply represents semantic meaning. Standard text embeddings use the outputs from hidden layers of a pretrained language model. In our method, we let a language model learn from the text and then literally pick its brain, taking the actual weights of the model's neurons to generate a vector. We call this representation of the text a neural embedding. We confirm the ability of this representation to reflect semantics of the text by an analysis of its behavior on several datasets, and by a comparison of neural embedding with state of the art sentence embeddings.
