论文标题

通过非IID数据,通过非IID数据改进联合学习的沟通效率扩散策略

Communication-Efficient Diffusion Strategy for Performance Improvement of Federated Learning with Non-IID Data

论文作者

Ahn, Seyoung, Kim, Soohyeong, Kwon, Yongseok, Park, Joohan, Youn, Jiseung, Cho, Sunghyun

论文摘要

在6G移动通信系统中,已经标准化了各种基于AI的网络功能和应用程序。联合学习(FL)被用作6G系统的核心学习体系结构,以避免移动用户数据中的隐私泄漏。但是,在FL中,具有非独立和相同分布的(非IID)数据集的用户可能会恶化全局模型的性能,因为每个数据集的梯度的收敛方向是不同的,从而诱发了权重差异问题。为了解决这个问题,我们提出了一种新颖的机器学习扩散策略(ML)模型(FIDDIF),以通过非IID数据最大化全球模型的性能。 FedDif使本地模型通过通过设备到设备通信通过用户传递本地模型,可以在参数聚合之前学习不同的分布。此外,从理论上讲,我们证明了FedDif可以规避权重问题。基于这一理论,我们为ML模型提出了一种通信效率的扩散策略,可以通过拍卖理论决定学习绩效和交流成本之间的权衡。实验结果表明,FedDIF将TOP-1测试的准确性提高了34.89 \%,并将通信成本降至最高63.49%。

In 6G mobile communication systems, various AI-based network functions and applications have been standardized. Federated learning (FL) is adopted as the core learning architecture for 6G systems to avoid privacy leakage from mobile user data. However, in FL, users with non-independent and identically distributed (non-IID) datasets can deteriorate the performance of the global model because the convergence direction of the gradient for each dataset is different, thereby inducing a weight divergence problem. To address this problem, we propose a novel diffusion strategy for machine learning (ML) models (FedDif) to maximize the performance of the global model with non-IID data. FedDif enables the local model to learn different distributions before parameter aggregation by passing the local models through users via device-to-device communication. Furthermore, we theoretically demonstrate that FedDif can circumvent the weight-divergence problem. Based on this theory, we propose a communication-efficient diffusion strategy for ML models that can determine the trade-off between learning performance and communication cost using auction theory. The experimental results show that FedDif improves the top-1 test accuracy by up to 34.89\% and reduces communication costs by 14.6% to a maximum of 63.49%.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源