论文标题
通过检索软提示，有效地增强了教学后的零射击性能
Efficiently Enhancing Zero-Shot Performance of Instruction Following Model via Retrieval of Soft Prompt
论文作者
论文摘要
通过缩放训练数据集的总数或模型大小，提高指令跟随模型的零射击性能需要进行大量计算。在这项工作中，我们探讨了如何通过迅速调整获得的软提示的检索如何有效地帮助硬提示以零击任务概括。具体来说，我们通过提示调整，存储用提示嵌入的训练实例的样本来训练每个提示符的软提示嵌入，并在推理过程中检索最接近查询实例的培训实例的相应提示嵌入。尽管仅添加0.007％的附加参数，但软提示的检索通过在11个数据集中的10个数据集中胜过10个，并提高了T0的表现，并提高了T0的表现，并提高了Big-benchmark上T0的平均准确性，并提高了2.39％的点。另外，我们报告了一个有趣的发现，检索在类似的答案选择格式训练的源嵌入比相似的任务类型的嵌入更为重要。
Enhancing the zero-shot performance of instruction-following models requires heavy computation, either by scaling the total number of training datasets or the model size. In this work, we explore how retrieval of soft prompts obtained through prompt tuning can efficiently assist hard prompts in zero-shot task generalization. Specifically, we train soft prompt embeddings for each prompt through prompt tuning, store the samples of the training instances mapped with the prompt embeddings, and retrieve the corresponding prompt embedding of the training instance closest to the query instance during inference. While only adding 0.007% additional parameters, retrieval of soft prompt enhances the performance of T0 on unseen tasks by outperforming it on 10 out of 11 datasets as well as improving the mean accuracy of T0 on BIG-bench benchmark by 2.39% points. Also, we report an interesting finding that retrieving source embeddings trained on similar answer choice formats is more important than those on similar task types.
