aiqbench analysis
This research introduces RNC-LM, an optimizer that corrects geometric inconsistencies in the Levenberg-Marquardt method by following curved trajectories in Riemann normal coordinates. By decoupling curve length from damping parameters, the approach achieves significant speedups and lower error rates on physics-informed neural networks compared to standard optimizers. The work establishes finite-step geometric realization as a critical design principle for scientific machine learning tasks involving nonlinear least-squares objectives.
Higher-Order Geometric Updates for Levenberg-Marquardt Method via Riemann Normal Coordinates
arXiv:2607.07623v2 Announce Type: replace Abstract: Nonlinear least-squares objectives form the foundation of scientific machine-learning tasks, yet even curvature-aware optimizers remain geometrically inconsistent at finite step sizes: Levenberg-Marquardt (LM) derives its direction from local Riemannian metrics but realizes it as a straight parameter update. Here we introduce RNC-LM, which carries the LM direction along a locally constructed curved trajectory in Riemann normal coordinates. A recursive reformulation of the geodesic equation generates arbitrary finite-order corrections while reusing the same damped Gauss-Newton matrix factorization, and curve length is controlled separately from damping. On a reaction-diffusion physics-informed neural network benchmark, RNC-LM reduces relative $L^2$ errors below $8\times10^{-3}$, whereas L-BFGS, LM and LM with geodesic acceleration remain near one. On a large-scale machine-learning potential fitting task with 985,160 configurations, fourth-order RNC-LM reaches a fixed training-error target with a $34\times$ wall-clock speedup over LM. These results establish finite-step geometric realization as a distinct optimizer design principle.
Abstract:Nonlinear least-squares objectives form the foundation of scientific machine-learning tasks, yet even curvature-aware optimizers remain geometrically inconsistent at finite step sizes: Levenberg-Marquardt (LM) derives its direction from local Riemannian metrics but realizes it as a straight parameter update. Here we introduce RNC-LM, which carries the LM direction along a locally constructed curved trajectory in Riemann normal coordinates. A recursive reformulation of the geodesic equation generates arbitrary finite-order corrections while reusing the same damped Gauss-Newton matrix factorization, and curve length is controlled separately from damping. On a reaction-diffusion physics-informed neural network benchmark, RNC-LM reduces relative $L^2$ errors below $8\times10^{-3}$, whereas L-BFGS, LM and LM with geodesic acceleration remain near one. On a large-scale machine-learning potential fitting task with 985,160 configurations, fourth-order RNC-LM reaches a fixed training-error target with a $34\times$ wall-clock speedup over LM. These results establish finite-step geometric realization as a distinct optimizer design principle.
Machine Learning (cs.LG); Numerical Analysis (math.NA); Chemical Physics (physics.chem-ph); Computational Physics (physics.comp-ph); Data Analysis, Statistics and Probability (physics.data-an)
arXiv:2607.07623 [cs.LG]
(or arXiv:2607.07623v2 [cs.LG] for this version)
Full text under the publisher's open licence · 171 words
