论文标题
探索适应协议的设计，以改善概括和机器学习安全
Exploring the Design of Adaptation Protocols for Improved Generalization and Machine Learning Safety
论文作者
论文摘要
虽然直接进行微调（FT）大规模的，在特定于任务数据上进行了预审计的模型是众所周知的，这引起了强大的分配任务绩效，但最近的工作表明，在ft之前的线性探测（LP）等不同的适应方案可以改善分布外的通用。但是，此类适应协议的设计空间仍未探索，并且对此类协议的评估主要集中在分配转移上。因此，在这项工作中，我们评估了跨分布转移和机器学习安全指标的共同适应协议（例如，异常检测，校准，对腐败的鲁棒性）。我们发现，协议引起了不同的权衡，这些权衡在事先评估中并不明显。此外，我们证明，适当的数据增强和协议可以大大减轻这种权衡。最后，我们假设并从经验上看到，使用LP期间使用硬度促进硬度的增强功能，然后使用增强功能对ft进行ft可能对缓解权衡特别有效。
While directly fine-tuning (FT) large-scale, pretrained models on task-specific data is well-known to induce strong in-distribution task performance, recent works have demonstrated that different adaptation protocols, such as linear probing (LP) prior to FT, can improve out-of-distribution generalization. However, the design space of such adaptation protocols remains under-explored and the evaluation of such protocols has primarily focused on distribution shifts. Therefore, in this work, we evaluate common adaptation protocols across distributions shifts and machine learning safety metrics (e.g., anomaly detection, calibration, robustness to corruptions). We find that protocols induce disparate trade-offs that were not apparent from prior evaluation. Further, we demonstrate that appropriate pairing of data augmentation and protocol can substantially mitigate this trade-off. Finally, we hypothesize and empirically see that using hardness-promoting augmentations during LP and then FT with augmentations may be particularly effective for trade-off mitigation.
