论文标题
通过联邦蒸馏而对褪色渠道进行合作学习
Cooperative Learning via Federated Distillation over Fading Channels
论文作者
论文摘要
分布式机器学习的合作培训方法通常基于本地梯度或本地模型参数的交换。后一种方法称为联合学习（FL）。最近提出了一种替代通信开销（FD）的替代解决方案，该解决方案仅交换平均模型输出。尽管先前的工作研究了无线褪色渠道的FL实现，但在这里，我们提出了针对FD的无线协议，并为其增强版本的版本提供了一个利用离线通信阶段来传达``混合''协变量向量的。所提出的实现由基于单独的源通道编码和基于模拟联合源通道编码的空中计算策略的数字方案的不同组合组合。结果表明，在有限的光谱资源的存在下，增强的版本FD有可能显着胜过FL。
Cooperative training methods for distributed machine learning are typically based on the exchange of local gradients or local model parameters. The latter approach is known as Federated Learning (FL). An alternative solution with reduced communication overhead, referred to as Federated Distillation (FD), was recently proposed that exchanges only averaged model outputs. While prior work studied implementations of FL over wireless fading channels, here we propose wireless protocols for FD and for an enhanced version thereof that leverages an offline communication phase to communicate ``mixed-up'' covariate vectors. The proposed implementations consist of different combinations of digital schemes based on separate source-channel coding and of over-the-air computing strategies based on analog joint source-channel coding. It is shown that the enhanced version FD has the potential to significantly outperform FL in the presence of limited spectral resources.
