论文标题
贝叶斯及时学习图像语言模型概括
Bayesian Prompt Learning for Image-Language Model Generalization
论文作者
论文摘要
基本的图像语言模型由于通过及时学习对下游任务的有效调整而产生了巨大的兴趣。迅速学习将部分语言模型的一部分视为可训练的同时，同时冻结其余部分，并优化了经验风险最小化目标。但是，已知经验风险最小化会遭受分配变化的损害，这些变化损害了在训练期间提示未见的普遍性。通过利用贝叶斯方法的正则化能力，我们从贝叶斯的角度提示学习并将其作为变异的推断问题。我们的方法使及时空间的正规规范，减少了对可见提示的过度拟合，并改善了对看不见的提示的及时概括。我们的框架是通过以概率方式对输入提示空间进行建模来实现的，作为先验分布，这使我们的建议与图像无条件或条件的及时学习方法兼容。我们在15个基准上进行了经验证明，贝叶斯及时学习提供了适当的及时空间的覆盖范围，可防止学习虚假功能，并利用可转移的不变功能。这也可以更好地概括看不见的提示，即使在不同的数据集和域之间也可以更好地概括。代码可用：https：//github.com/saic-fi/bayesian-prompt-learning
Foundational image-language models have generated considerable interest due to their efficient adaptation to downstream tasks by prompt learning. Prompt learning treats part of the language model input as trainable while freezing the rest, and optimizes an Empirical Risk Minimization objective. However, Empirical Risk Minimization is known to suffer from distributional shifts which hurt generalizability to prompts unseen during training. By leveraging the regularization ability of Bayesian methods, we frame prompt learning from the Bayesian perspective and formulate it as a variational inference problem. Our approach regularizes the prompt space, reduces overfitting to the seen prompts and improves the prompt generalization on unseen prompts. Our framework is implemented by modeling the input prompt space in a probabilistic manner, as an a priori distribution which makes our proposal compatible with prompt learning approaches that are unconditional or conditional on the image. We demonstrate empirically on 15 benchmarks that Bayesian prompt learning provides an appropriate coverage of the prompt space, prevents learning spurious features, and exploits transferable invariant features. This results in better generalization of unseen prompts, even across different datasets and domains. Code available at: https://github.com/saic-fi/Bayesian-Prompt-Learning
