论文标题
没有信任的服务器的私人非凸线联合学习
Private Non-Convex Federated Learning Without a Trusted Server
论文作者
论文摘要
我们研究了联合学习（FL），尤其是Cross-Silo FL-具有不信任服务器或其他筒仓的人的非凸损失功能和数据。在这种情况下，即使服务器或其他筒仓作为对抗性窃听者，每个孤岛（例如医院）也必须保护每个人数据的隐私（例如患者的病历）。为此，我们考虑使用SILO Inter-Silo记录级（ISRL）差异隐私（DP），这需要筒仓〜$ i $的通信来满足记录/项目级DP。我们提出了具有异质性（非I.I.D。）孤岛数据和两类Lipschitz连续损耗功能的新型ISRL-DP算法：首先，我们考虑满足近端Polyak-lojasiewicz（PL）不平等的损失，这是经典PL条件的扩展到约束条件的扩展。与我们的结果相反，先前的作品仅考虑了Lipschitz PL损失的不受约束的私人优化，这排除了最有趣的PL损失，例如强烈凸出问题和线性/逻辑回归。我们的算法几乎达到了ISRL-DP FL的最佳凸，均匀（I.I.D。）的最佳速率，而无需假设凸度或I.I.D.数据。其次，我们给出了非凸线非平滑损失功能的第一个私人算法。我们的公用事业界限甚至可以改善最先进的范围，以实现平稳损失。我们用下限补充上限。此外，我们还提供Shuffle DP（SDP）算法，这些算法在更实际的信任假设下对最新的中央DP算法进行了改进。数值实验表明，对于大多数隐私级别，我们的算法比基线具有更好的准确性。所有代码均可在以下网址公开获取：https：//github.com/ghafeleb/private-nonconvex-federated-learning-without-a-trusted-server。
We study federated learning (FL) -- especially cross-silo FL -- with non-convex loss functions and data from people who do not trust the server or other silos. In this setting, each silo (e.g. hospital) must protect the privacy of each person's data (e.g. patient's medical record), even if the server or other silos act as adversarial eavesdroppers. To that end, we consider inter-silo record-level (ISRL) differential privacy (DP), which requires silo~$i$'s communications to satisfy record/item-level DP. We propose novel ISRL-DP algorithms for FL with heterogeneous (non-i.i.d.) silo data and two classes of Lipschitz continuous loss functions: First, we consider losses satisfying the Proximal Polyak-Lojasiewicz (PL) inequality, which is an extension of the classical PL condition to the constrained setting. In contrast to our result, prior works only considered unconstrained private optimization with Lipschitz PL loss, which rules out most interesting PL losses such as strongly convex problems and linear/logistic regression. Our algorithms nearly attain the optimal strongly convex, homogeneous (i.i.d.) rate for ISRL-DP FL without assuming convexity or i.i.d. data. Second, we give the first private algorithms for non-convex non-smooth loss functions. Our utility bounds even improve on the state-of-the-art bounds for smooth losses. We complement our upper bounds with lower bounds. Additionally, we provide shuffle DP (SDP) algorithms that improve over the state-of-the-art central DP algorithms under more practical trust assumptions. Numerical experiments show that our algorithm has better accuracy than baselines for most privacy levels. All the codes are publicly available at: https://github.com/ghafeleb/Private-NonConvex-Federated-Learning-Without-a-Trusted-Server.
