论文标题
一种用于协作分布式学习的信息理论方法，沟通有限
An Information-theoretic Method for Collaborative Distributed Learning with Limited Communication
论文作者
论文摘要
在本文中，我们研究了分布式学习框架下的信息传输问题，在该框架下，每个工人节点仅允许传输$ m $维的统计量，以改善目标节点的学习结果。具体而言，我们在大型样本量的状态下评估了相应的预期人口风险（EPR）。我们证明可以增强性能，因为传输统计数据有助于估计EPR Norm矩阵测量的均方根误差下的基本分布。因此，传输统计量对应于该矩阵的特征向量，所需的传输将这些特征向量分配在统计数据中，从而使EPR最小。此外，我们为单节点和两节点传输提供了所需统计数据的分析解决方案，其中给出了几何解释来解释特征向量选择。对于一般情况，可以根据节点分区开发可以输出分配解决方案的有效算法。
In this paper, we study the information transmission problem under the distributed learning framework, where each worker node is merely permitted to transmit a $m$-dimensional statistic to improve learning results of the target node. Specifically, we evaluate the corresponding expected population risk (EPR) under the regime of large sample sizes. We prove that the performance can be enhanced since the transmitted statistics contribute to estimating the underlying distribution under the mean square error measured by the EPR norm matrix. Accordingly, the transmitted statistics correspond to the eigenvectors of this matrix, and the desired transmission allocates these eigenvectors among the statistics such that the EPR is minimal. Moreover, we provide the analytical solution of the desired statistics for single-node and two-node transmission, where a geometrical interpretation is given to explain the eigenvector selection. For the general case, an efficient algorithm that can output the allocation solution is developed based on the node partitions.
