论文标题
关于上下文嵌入的调查
A Survey on Contextual Embeddings
论文作者
论文摘要
诸如Elmo和Bert之类的上下文嵌入超越了诸如Word2Vec之类的全球单词表示，并在各种自然语言处理任务上实现了突破性的性能。上下文嵌入基于其上下文分配每个单词的表示，从而捕获了各种上下文中单词的用途，并编码跨语言传输的知识。在本调查中，我们回顾了现有的上下文嵌入模型，跨语性多语言预训练，上下文嵌入在下游任务中的应用，模型压缩和模型分析。
Contextual embeddings, such as ELMo and BERT, move beyond global word representations like Word2Vec and achieve ground-breaking performance on a wide range of natural language processing tasks. Contextual embeddings assign each word a representation based on its context, thereby capturing uses of words across varied contexts and encoding knowledge that transfers across languages. In this survey, we review existing contextual embedding models, cross-lingual polyglot pre-training, the application of contextual embeddings in downstream tasks, model compression, and model analyses.
