aiqbench analysis
This tutorial explores using differentiable programming frameworks like PyTorch and JAX to end-to-end train first-order optimization algorithms, shifting the paradigm from manual design to learned algorithmic adaptation. By embedding methods such as ADMM within these systems, researchers can leverage automatic differentiation to improve convergence rates and solution quality for massive-scale problems. This approach represents a significant methodological advance in machine learning infrastructure where the core innovation lies in optimizing the optimization process itself.
Learning to Optimize by Differentiable Programming
arXiv:2601.16510v5 Announce Type: replace-cross Abstract: Solving massive-scale optimization problems requires scalable first-order methods with low per-iteration cost. This tutorial highlights a shift in optimization: using differentiable programming not only to execute algorithms but to learn how to design them. Modern frameworks such as PyTorch, TensorFlow, and JAX enable this paradigm through efficient automatic differentiation. Embedding first-order methods within these systems allows end-to-end training that improves convergence and solution quality. Guided by Fenchel-Rockafellar duality, the tutorial demonstrates how duality-informed iterative schemes such as the alternating direction method of multipliers, and the primal-dual hybrid gradient can be learned and adapted through representative case studies.
Abstract:Solving massive-scale optimization problems requires scalable first-order methods with low per-iteration cost. This tutorial highlights a shift in optimization: using differentiable programming not only to execute algorithms but to learn how to design them. Modern frameworks such as PyTorch, TensorFlow, and JAX enable this paradigm through efficient automatic differentiation. Embedding first-order methods within these systems allows end-to-end training that improves convergence and solution quality. Guided by Fenchel-Rockafellar duality, the tutorial demonstrates how duality-informed iterative schemes such as the alternating direction method of multipliers, and the primal-dual hybrid gradient can be learned and adapted through representative case studies.
Mathematical Software (cs.MS); Machine Learning (cs.LG); Optimization and Control (math.OC)
arXiv:2601.16510 [cs.MS]
(or arXiv:2601.16510v5 [cs.MS] for this version)
https://doi.org/10.48550/arXiv.2601.16510 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 196 words
Related coverage
The Elements of Differentiable Programming
arXiv · Aug 5, 2026
News
Quantum Computing Report · Jun 7, 2026
dOPT: Differentiating Conic Optimization via Geometric Reduction
arXiv · Sep 30, 2026
Compile Once, Differentiate Everywhere: A Differentiable Meta-Circular Interpreter
arXiv · Jun 10, 2026