论文标题
Graphnorm：一种加速图神经网络训练的原则方法
GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training
论文作者
论文摘要
已知归一化有助于优化深层神经网络。奇怪的是，不同的体系结构需要专门的归一化方法。在本文中，我们研究了什么归一化对图神经网络有效（GNNS）。首先，我们适应并评估从其他域到GNN的现有方法。与batchnorm和Layernorm相比，使用激型型实现更快的收敛速度。我们通过证明Instancenorm充当GNN的预处理，提供了一种解释，但是由于图数据集中的批处理噪声较重，这种预处理效果较弱。其次，我们表明Instancenorm中的移位操作导致GNNS的表达性降解，用于高正规图。我们通过提出可学习的转变来解决这个问题。从经验上，使用其他归一化的GNN与GNN相比，具有石墨型的GNN会更快。 Graphnorm还改善了GNN的概括，在图形分类基准上取得了更好的性能。
Normalization is known to help the optimization of deep neural networks. Curiously, different architectures require specialized normalization methods. In this paper, we study what normalization is effective for Graph Neural Networks (GNNs). First, we adapt and evaluate the existing methods from other domains to GNNs. Faster convergence is achieved with InstanceNorm compared to BatchNorm and LayerNorm. We provide an explanation by showing that InstanceNorm serves as a preconditioner for GNNs, but such preconditioning effect is weaker with BatchNorm due to the heavy batch noise in graph datasets. Second, we show that the shift operation in InstanceNorm results in an expressiveness degradation of GNNs for highly regular graphs. We address this issue by proposing GraphNorm with a learnable shift. Empirically, GNNs with GraphNorm converge faster compared to GNNs using other normalization. GraphNorm also improves the generalization of GNNs, achieving better performance on graph classification benchmarks.
