论文标题
使用暹罗网络和标签调整几乎没有学习
Few-Shot Learning with Siamese Networks and Label Tuning
论文作者
论文摘要
我们研究了几乎没有或没有培训数据的构建文本分类器的问题，通常称为零和很少的文本分类。近年来，已经发现了一种基于神经文本索引模型的方法，可以在各种任务上产生良好的结果。在这项工作中，我们表明，通过适当的预训练，嵌入文本和标签的暹罗网络提供了竞争性的选择。这些模型可以大大降低推理成本：标签数而不是线性的恒定。此外，我们介绍了标签调整，这是一种简单且计算上有效的方法，可以通过仅更改标签嵌入来以几次摄入设置来调整模型。在给出比模型微调较低的性能的同时，这种方法具有架构优势，即单个编码器可以通过许多不同的任务共享。
We study the problem of building text classifiers with little or no training data, commonly known as zero and few-shot text classification. In recent years, an approach based on neural textual entailment models has been found to give strong results on a diverse range of tasks. In this work, we show that with proper pre-training, Siamese Networks that embed texts and labels offer a competitive alternative. These models allow for a large reduction in inference cost: constant in the number of labels rather than linear. Furthermore, we introduce label tuning, a simple and computationally efficient approach that allows to adapt the models in a few-shot setup by only changing the label embeddings. While giving lower performance than model fine-tuning, this approach has the architectural advantage that a single encoder can be shared by many different tasks.
