论文标题
优化反向模式自动差异化的稀疏矩阵操作
Optimized Sparse Matrix Operations for Reverse Mode Automatic Differentiation
论文作者
论文摘要
稀疏的矩阵表示在计算科学和机器学习中无处不在，与具有局部连接的问题相比，与密集表示相比，计算时间大幅减少。但是，在诸如Pytorch之类的领先ML框架中的稀疏表示不完整，但支持自动分化和GPU加速度缺失。在这项工作中，我们介绍了基于CSR的稀疏矩阵包装器的Pytorch稀疏矩阵包装器，以及用于基本矩阵操作的CUDA加速度以及自动可不同的性能。我们还向最终的稀疏内核提出了优化问题的几种应用，这表明实施和绩效测量易于实施，而不是其密集的对应物。
Sparse matrix representations are ubiquitous in computational science and machine learning, leading to significant reductions in compute time, in comparison to dense representation, for problems that have local connectivity. The adoption of sparse representation in leading ML frameworks such as PyTorch is incomplete, however, with support for both automatic differentiation and GPU acceleration missing. In this work, we present an implementation of a CSR-based sparse matrix wrapper for PyTorch with CUDA acceleration for basic matrix operations, as well as automatic differentiability. We also present several applications of the resulting sparse kernels to optimization problems, demonstrating ease of implementation and performance measurements versus their dense counterparts.
