aiqbench analysis
This research introduces a novel optimizer for scientific machine learning that integrates momentum and adaptive mobility into the pullback-corrected scalar auxiliary variable framework. By formulating the update as a single implicit solve, the method achieves significant error reduction in physics-informed neural networks while maintaining rigorous stability guarantees. The approach represents a substantive advancement in optimization algorithms specifically tailored for complex multi-term loss landscapes common in scientific computing.
A pullback-corrected scalar auxiliary variable optimizer with momentum and adaptive mobility
arXiv:2609.13569v1 Announce Type: cross Abstract: Objectives in scientific machine learning are often prescribed as a sum of several terms, such as the residual, boundary, initial, and data losses of a physics-informed neural network. In the pullback-corrected scalar auxiliary variable (PB--SAV) method, one scalar tracks the shifted objective while the component gradients build a positive semidefinite curvature correction of rank at most the number of components. We carry that correction into an optimizer with momentum and an adaptive mobility, applying it to the gradient and the stored momentum in a single implicit solve. A mobility that is nonincreasing in the Loewner order yields an exact modified energy law, covering Euclidean and AMSGrad-type choices; the corresponding identity for momentum appended after the solve carries a cross term of indefinite sign. For a fixed mobility we give a necessary and sufficient condition for local stability at a stationary point, depending on the Hessian minus twice the correction, and show that it also gives local geometric convergence for every scalar relaxation sequence. The implicit solve reduces to a dense system whose order is the number of components. In the forward Burgers comparison, four components reduce the mean tail objective by 64.7% and the final solution error by 50.2% relative to one component at the same learning rate and momentum settings.
Abstract:Objectives in scientific machine learning are often prescribed as a sum of several terms, such as the residual, boundary, initial, and data losses of a physics-informed neural network. In the pullback-corrected scalar auxiliary variable (PB--SAV) method, one scalar tracks the shifted objective while the component gradients build a positive semidefinite curvature correction of rank at most the number of components. We carry that correction into an optimizer with momentum and an adaptive mobility, applying it to the gradient and the stored momentum in a single implicit solve. A mobility that is nonincreasing in the Loewner order yields an exact modified energy law, covering Euclidean and AMSGrad-type choices; the corresponding identity for momentum appended after the solve carries a cross term of indefinite sign. For a fixed mobility we give a necessary and sufficient condition for local stability at a stationary point, depending on the Hessian minus twice the correction, and show that it also gives local geometric convergence for every scalar relaxation sequence. The implicit solve reduces to a dense system whose order is the number of components. In the forward Burgers comparison, four components reduce the mean tail objective by 64.7% and the final solution error by 50.2% relative to one component at the same learning rate and momentum settings.
Machine Learning (stat.ML); Machine Learning (cs.LG)
arXiv:2609.13569 [stat.ML]
(or arXiv:2609.13569v1 [stat.ML] for this version)
https://doi.org/10.48550/arXiv.2609.13569 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 308 words
Related coverage
Performance Profiling on AMD GPUs – Part 5: Profiling-Driven Kernel Optimization with an AI Code-Assist Tool
AMD · Jul 16, 2026
A new pulse design cuts errors in hybrid qubit-oscillator control
Quantum Zeitgeist · Oct 15, 2026
Lattice-Based Cryptography Explained
Quantum Zeitgeist · Aug 15, 2026
Momentum as Residual-Driven Multiplier Correction for Deep Learning Optimization
arXiv · Sep 1, 2026