论文标题
Modeldiff：比较学习算法的框架
ModelDiff: A Framework for Comparing Learning Algorithms
论文作者
论文摘要
我们研究了（学习）算法比较的问题，目的是在使用两种不同学习算法的模型之间找到差异。我们首先将这个目标形式化为一个找到区分特征转换的一个目标之一，即，输入转换会改变用一种学习算法训练的模型的预测，而不是另一种。然后，我们提出了Modeldiff，一种利用Datamodels框架（Ilyas等，2022）的方法，根据他们如何使用培训数据来比较学习算法。我们通过三个案例研究证明了模仿者，比较了通过/没有数据增强的模型，使用/没有预训练的模型以及不同的SGD超参数。我们的代码可从https://github.com/madrylab/modeldiff获得。
We study the problem of (learning) algorithm comparison, where the goal is to find differences between models trained with two different learning algorithms. We begin by formalizing this goal as one of finding distinguishing feature transformations, i.e., input transformations that change the predictions of models trained with one learning algorithm but not the other. We then present ModelDiff, a method that leverages the datamodels framework (Ilyas et al., 2022) to compare learning algorithms based on how they use their training data. We demonstrate ModelDiff through three case studies, comparing models trained with/without data augmentation, with/without pre-training, and with different SGD hyperparameters. Our code is available at https://github.com/MadryLab/modeldiff .
