aiqbench analysis
torch-sla addresses a critical gap in the PyTorch ecosystem by providing a unified, autograd-aware library for differentiable sparse linear algebra, enabling scalable scientific machine learning workloads. The library unifies five interchangeable backends ranging from CPU to GPU with automatic dispatch and supports distributed multi-GPU execution via domain decomposition. This infrastructure advancement is significant for researchers and engineers building large-scale models that require efficient, differentiable sparse solver operations.
torch-sla: Differentiable Sparse Linear Algebra with Adjoint Solvers and Sparse Tensor Parallelism for PyTorch
arXiv:2601.13994v3 Announce Type: replace-cross Abstract: Differentiable sparse linear algebra is foundational for scientific machine learning, yet PyTorch lacks a unified library for it: torch.sparse provides only low-level kernels and a non-differentiable, CPU-only spsolve, and torch.linalg is dense-only. We present torch-sla, an open-source library that fills this gap. It exposes a single autograd-aware API for direct, iterative, nonlinear, and eigenvalue solvers across five interchangeable backends -- SciPy and Eigen on CPU, cuDSS, CuPy, and a PyTorch-native iterative solver on GPU -- with automatic dispatch by device and problem size. The library further supports batched solves over shared or distinct sparsity patterns and distributed multi-GPU execution via domain decomposition with halo exchange. These capabilities are made scalable by an O(1)-graph adjoint differentiation framework and an autograd-compatible distributed halo-exchange layer. The library is available at https://www.torchsla.com/.
Abstract:Differentiable sparse linear algebra is foundational for scientific machine learning, yet PyTorch lacks a unified library for it: this http URL provides only low-level kernels and a non-differentiable, CPU-only spsolve, and this http URL is dense-only. We present torch-sla, an open-source library that fills this gap. It exposes a single autograd-aware API for direct, iterative, nonlinear, and eigenvalue solvers across five interchangeable backends -- SciPy and Eigen on CPU, cuDSS, CuPy, and a PyTorch-native iterative solver on GPU -- with automatic dispatch by device and problem size. The library further supports batched solves over shared or distinct sparsity patterns and distributed multi-GPU execution via domain decomposition with halo exchange. These capabilities are made scalable by an O(1)-graph adjoint differentiation framework and an autograd-compatible distributed halo-exchange layer. The library is available at this https URL.
Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI)
arXiv:2601.13994 [cs.DC]
(or arXiv:2601.13994v3 [cs.DC] for this version)
Full text under the publisher's open licence · 151 words
Related coverage
SALTO: Solver-Anchored Lifted Transfer Operators for Nonlinear Event Inference
arXiv · Oct 16, 2026
DFSC: Error-Controlled Differentiable Mittag-Leffler Propagation for Fractional Scientific Machine Learning
arXiv · Aug 3, 2026
MixedComplementarityProblems.jl: A Fast, Batched, Open-Source Interior Point Solver for Mixed Complementarity Problems
arXiv · Aug 4, 2026
Finite Entanglement Scaling Origin Linked to System Perturbations
Quantum Zeitgeist · Aug 14, 2026