论文标题
基于影响的图形神经网络的迷你批次
Influence-Based Mini-Batching for Graph Neural Networks
论文作者
论文摘要
将图神经网络用于大图很具有挑战性，因为没有明确构建迷你批次的方法。为了解决这个问题，以前的方法依赖于采样或图形聚类。尽管这些方法通常会导致良好的培训融合，但由于昂贵的随机数据访问，它们引入了大量的间接费用，并且在推断过程中表现不佳。相反，在这项工作中，我们专注于推断期间的模型行为。从理论上讲，我们通过最大化节点对输出的影响评分来建模。当我们不知道受过训练的模型时，这种公式会导致输出的最佳近似。我们称之为最终方法基于影响力的迷你批次（IBMB）。与以前达到相似精度的方法相比，IBMB的推断最多可加速130倍。值得注意的是，通过自适应优化和正确的培训时间表，IBMB也可以基本上加速培训，这要归功于预先计算的批次和连续的内存访问。与以前的方法相比，每个时期的训练速度高达18倍，每个运行时的收敛速度最高高17倍。
Using graph neural networks for large graphs is challenging since there is no clear way of constructing mini-batches. To solve this, previous methods have relied on sampling or graph clustering. While these approaches often lead to good training convergence, they introduce significant overhead due to expensive random data accesses and perform poorly during inference. In this work we instead focus on model behavior during inference. We theoretically model batch construction via maximizing the influence score of nodes on the outputs. This formulation leads to optimal approximation of the output when we do not have knowledge of the trained model. We call the resulting method influence-based mini-batching (IBMB). IBMB accelerates inference by up to 130x compared to previous methods that reach similar accuracy. Remarkably, with adaptive optimization and the right training schedule IBMB can also substantially accelerate training, thanks to precomputed batches and consecutive memory accesses. This results in up to 18x faster training per epoch and up to 17x faster convergence per runtime compared to previous methods.
