论文标题
转移学习中的一类几何结构：最小值和最佳性
A Class of Geometric Structures in Transfer Learning: Minimax Bounds and Optimality
论文作者
论文摘要
我们研究了转移学习的问题，观察到以前了解其信息理论限制的努力并不能完全利用来源和目标域的几何结构。相反，我们的研究首先说明了将自然几何结构纳入线性回归模型的好处，该模型与两个域的革兰氏矩阵形成的广义特征值问题相对应。接下来，我们建立一个有限的最小值下限，提出了一个精致的模型插值估计器，该估计器具有匹配的上限，然后将我们的框架扩展到多个源域和广义线性模型。令人惊讶的是，只要可以在源和目标参数之间的距离上提供信息，就不会发生负转移。仿真研究表明，我们提出的插值估计器在中等和高维环境中的最先进传递学习方法优于最先进的转移学习方法。
We study the problem of transfer learning, observing that previous efforts to understand its information-theoretic limits do not fully exploit the geometric structure of the source and target domains. In contrast, our study first illustrates the benefits of incorporating a natural geometric structure within a linear regression model, which corresponds to the generalized eigenvalue problem formed by the Gram matrices of both domains. We next establish a finite-sample minimax lower bound, propose a refined model interpolation estimator that enjoys a matching upper bound, and then extend our framework to multiple source domains and generalized linear models. Surprisingly, as long as information is available on the distance between the source and target parameters, negative-transfer does not occur. Simulation studies show that our proposed interpolation estimator outperforms state-of-the-art transfer learning methods in both moderate- and high-dimensional settings.
