论文标题
联合无监督的代表学习
Federated Unsupervised Representation Learning
论文作者
论文摘要
为了利用分布式边缘设备上的巨大未标记数据，我们在联合学习中提出了一个新问题，称为联邦无监督的表示学习（FURL），以在没有监督的情况下学习一个共同的表示模型，同时保留数据隐私。 FURL提出了两个新的挑战：（1）客户之间的数据分配转移（非IID分布）将使本地模型集中在不同的类别上，从而导致表示空间的不一致。 （2）如果没有FURL中客户之间的统一信息，客户的表示将被误解。为了应对这些挑战，我们提出了使用字典和对齐（FEDCA）算法的联合约束平均。 FEDCA由两个关键模块组成：（1）词典模块以汇总每个客户端的样本表示形式，并与所有客户端共享表示表示空间的一致性以及（2）对齐模块，以对齐每个客户端在公共数据中训练的基本模型上的表示。我们对当地模型培训采取对比损失。通过在IID和非IID设置中具有三个评估方案的广泛实验，我们证明了FedCa的表现优于所有基准，其边缘显着。
To leverage enormous unlabeled data on distributed edge devices, we formulate a new problem in federated learning called Federated Unsupervised Representation Learning (FURL) to learn a common representation model without supervision while preserving data privacy. FURL poses two new challenges: (1) data distribution shift (Non-IID distribution) among clients would make local models focus on different categories, leading to the inconsistency of representation spaces. (2) without the unified information among clients in FURL, the representations across clients would be misaligned. To address these challenges, we propose Federated Constrastive Averaging with dictionary and alignment (FedCA) algorithm. FedCA is composed of two key modules: (1) dictionary module to aggregate the representations of samples from each client and share with all clients for consistency of representation space and (2) alignment module to align the representation of each client on a base model trained on a public data. We adopt the contrastive loss for local model training. Through extensive experiments with three evaluation protocols in IID and Non-IID settings, we demonstrate that FedCA outperforms all baselines with significant margins.
