aiqbench analysis
This paper introduces Split-Client, a task-parallel architecture for second-order optimization that decouples Hessian factorization from gradient updates to reduce wall-clock time. By proving stationarity bounds dependent on the average age of curvature data rather than maximum delay, it achieves performance comparable to optimally tuned Lazy Hessian methods without requiring manual tuning of reuse periods. The approach demonstrates significant speedups in practical scenarios, particularly at intermediate accuracy levels where computational overhead is most critical.
Second-Order Optimization with Asynchronous Curvature
arXiv:2510.15714v4 Announce Type: replace-cross Abstract: Second-order methods make fast progress per iteration, but forming and factorizing the Hessian often dominates their wall-clock time. We study \emph{Split-Client}, a task-parallel design in which a curvature client factorizes Hessians in the background while a gradient client takes cubic-regularized steps with the most recently published matrix, so that curvature cost appears as a delay rather than as blocking time. For delayed exact Hessians, we prove a first-order stationarity bound that depends on the average age of the curvature rather than its maximum, using an overlap count and a weighted Hardy inequality that allows any bounded-error initial matrix; the analysis extends to subsampled and uniformly inexact curvature. Under a fixed-cost model, the resulting wall-clock bound matches the scaling of optimally tuned Lazy Hessian without choosing a reuse period. In experiments on two LIBSVM datasets, an implementation decreases the objective earlier in wall-clock time than Lazy and Vanilla cubic Newton, with the largest gains at intermediate accuracy.
Abstract:Second-order methods make fast progress per iteration, but forming and factorizing the Hessian often dominates their wall-clock time. We study \emph{Split-Client}, a task-parallel design in which a curvature client factorizes Hessians in the background while a gradient client takes cubic-regularized steps with the most recently published matrix, so that curvature cost appears as a delay rather than as blocking time. For delayed exact Hessians, we prove a first-order stationarity bound that depends on the average age of the curvature rather than its maximum, using an overlap count and a weighted Hardy inequality that allows any bounded-error initial matrix; the analysis extends to subsampled and uniformly inexact curvature. Under a fixed-cost model, the resulting wall-clock bound matches the scaling of optimally tuned Lazy Hessian without choosing a reuse period. In experiments on two LIBSVM datasets, an implementation decreases the objective earlier in wall-clock time than Lazy and Vanilla cubic Newton, with the largest gains at intermediate accuracy.
Optimization and Control (math.OC); Machine Learning (cs.LG)
arXiv:2510.15714 [math.OC]
(or arXiv:2510.15714v4 [math.OC] for this version)
https://doi.org/10.48550/arXiv.2510.15714 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 252 words
Related coverage
An Educational GEMM Ladder for Helios GPUs
AMD · Sep 14, 2026
Second-Order Path Kernel Interpolation Formulas in Machine Learning
arXiv · Jun 8, 2026
Why Muon Outperforms Adam: A Curvature Perspective
arXiv · Jun 5, 2026
TracingFlow: A Simulation-Free Trajectory Inference Framework Based on Second-Order Dynamics
arXiv · Aug 24, 2026