论文标题
BYGARS：拜占庭式SGD，有任意数量的攻击者
ByGARS: Byzantine SGD with Arbitrary Number of Attackers
论文作者
论文摘要
我们提出了两种新型随机梯度下降算法BYGARS和BYGARS ++，用于在存在任何数量的拜占庭对手的情况下进行分布式机器学习。在这些算法中，使用服务器上的辅助数据集计算工人的声誉得分。然后，该声誉评分用于汇总随机梯度下降的梯度。 BYGARS ++的计算复杂性与通常在每种迭代中具有附加的内部产品计算的通常分布式随机梯度下降法相同。我们表明，使用这些声誉得分进行梯度聚集是鲁棒的，对于任何数量的乘法噪声拜占庭对手，并使用两个timesscale随机近似理论来证明融合强烈凸出损失函数。我们证明了使用MNIST和CIFAR-10数据集对几乎所有最先进的拜占庭攻击的算法对非convex学习问题的有效性。我们还表明，所提出的算法同时对多种不同类型的攻击具有鲁棒性。
We propose two novel stochastic gradient descent algorithms, ByGARS and ByGARS++, for distributed machine learning in the presence of any number of Byzantine adversaries. In these algorithms, reputation scores of workers are computed using an auxiliary dataset at the server. This reputation score is then used for aggregating the gradients for stochastic gradient descent. The computational complexity of ByGARS++ is the same as the usual distributed stochastic gradient descent method with only an additional inner product computation in every iteration. We show that using these reputation scores for gradient aggregation is robust to any number of multiplicative noise Byzantine adversaries and use two-timescale stochastic approximation theory to prove convergence for strongly convex loss functions. We demonstrate the effectiveness of the algorithms for non-convex learning problems using MNIST and CIFAR-10 datasets against almost all state-of-the-art Byzantine attacks. We also show that the proposed algorithms are robust to multiple different types of attacks at the same time.
