论文标题
P^3排名：通过及时的学习和预先调查，减轻预训练和对微调的差距
P^3 Ranker: Mitigating the Gaps between Pre-training and Ranking Fine-tuning with Prompt-based Learning and Pre-finetuning
论文作者
论文摘要
与其他语言任务相比，应用预训练的语言模型（PLM）进行搜索排名通常需要更多的细微差别和培训信号。在本文中，我们确定并研究了预训练和排名微调之间的两个不匹配：关于培训目标和模型体系结构差异的培训模式差距，以及考虑排名中所需的知识与预训练期间所学的知识之间的差异的任务知识差距。为了减轻这些差距，我们提出了预先训练的，及时的且预先调查的神经排名（P^3等级）。 P^3等级器基于迅速的学习将排名任务转换为诸如模式之类的预训练，并使用预先调节来初始化中间监督任务的模型。 MS MARCO和ROBUST04上的实验显示了P^3等级的出色表现。分析表明，P^3排名者能够通过基于迅速的学习和检索必要的排名知识来更好地习惯于排名任务，从而在预先调节中收集到，从而获得了数据效率的PLM适应。我们的代码可在https://github.com/neuir/p3ranker上找到。
Compared to other language tasks, applying pre-trained language models (PLMs) for search ranking often requires more nuances and training signals. In this paper, we identify and study the two mismatches between pre-training and ranking fine-tuning: the training schema gap regarding the differences in training objectives and model architectures, and the task knowledge gap considering the discrepancy between the knowledge needed in ranking and that learned during pre-training. To mitigate these gaps, we propose Pre-trained, Prompt-learned and Pre-finetuned Neural Ranker (P^3 Ranker). P^3 Ranker leverages prompt-based learning to convert the ranking task into a pre-training like schema and uses pre-finetuning to initialize the model on intermediate supervised tasks. Experiments on MS MARCO and Robust04 show the superior performances of P^3 Ranker in few-shot ranking. Analyses reveal that P^3 Ranker is able to better accustom to the ranking task through prompt-based learning and retrieve necessary ranking-oriented knowledge gleaned in pre-finetuning, resulting in data-efficient PLM adaptation. Our code is available at https://github.com/NEUIR/P3Ranker.
