analiza aiqbench
This benchmark establishes a clearer baseline for optimizer performance by isolating algorithmic efficiency from hyperparameter tuning noise across diverse workloads. The findings highlight Schedule-Free AdamW as the current leader while noting that computationally intensive methods like matrix-preconditioned approaches still require optimization to match runtime competitiveness. For practitioners, this underscores the critical need for robust optimizers that generalize well without extensive manual tuning.
Towards Easier-to-Tune Training Algorithms: New Results from AlgoPerf
arXiv:2610.13036v1 Announce Type: new Abstract: Evaluating deep learning optimizer is challenging because confounding factors such as hyperparameter tuning and optimizer-model co-design often obscure true algorithmic performance. We present results of the updated AlgoPerf V1 benchmark, in which we evaluate a new set of training algorithms on wall-clock time-to-target across nine workloads on fixed hardware. This report focuses on the results of the benchmark's self-tuning ruleset, which requires submissions to use a single configuration across all workloads. Out of the 13 submissions in PyTorch and JAX, Schedule-Free AdamW ranked first. In addition to the wall-clock scores, we also analyze the trade-offs between step efficiency and wall-clock execution, revealing that matrix-preconditioned and two-loop methods achieve good sample efficiency but require more aggressive computational optimizations to be competitive in runtime. Finally, our hyperparameter saliency and retention analyses show that the learning rate and momentum are highly sensitive across domains and algorithms, highlighting the need for methods that are more robust against hyperparameter tuning and can generalize well to new tasks.
Authors:Priya Kasimbeg, Ahmed Khaled, Aahlad Chandrabhatta, Yifan Wang, Andy Liu, Niccolò Ajroldi, David Tweedle, Xingyu (Alice)Yang, Isaac Patole, Frank Schneider, George E. Dahl
Abstract:Evaluating deep learning optimizer is challenging because confounding factors such as hyperparameter tuning and optimizer-model co-design often obscure true algorithmic performance. We present results of the updated AlgoPerf V1 benchmark, in which we evaluate a new set of training algorithms on wall-clock time-to-target across nine workloads on fixed hardware. This report focuses on the results of the benchmark's self-tuning ruleset, which requires submissions to use a single configuration across all workloads. Out of the 13 submissions in PyTorch and JAX, Schedule-Free AdamW ranked first. In addition to the wall-clock scores, we also analyze the trade-offs between step efficiency and wall-clock execution, revealing that matrix-preconditioned and two-loop methods achieve good sample efficiency but require more aggressive computational optimizations to be competitive in runtime. Finally, our hyperparameter saliency and retention analyses show that the learning rate and momentum are highly sensitive across domains and algorithms, highlighting the need for methods that are more robust against hyperparameter tuning and can generalize well to new tasks.
Machine Learning (cs.LG)
arXiv:2610.13036 [cs.LG]
(or arXiv:2610.13036v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.13036 arXiv-issued DOI via DataCite (pending registration)
Text integral sub licența deschisă a publicației · 203 cuvinte
Subiecte conexe
[AINews] not much happened today
Latent Space · 18 sept. 2026
Expenditure Horizon: Measuring Optimization Ability, with an Application to NanoGPT
METR · 21 iul. 2026
Synthesis Superintelligence: from Semiconductors to Superconductors — Periodic Labs’ Liam Fedus and Ekin Dogus Cubuk
Latent Space · 8 oct. 2026
How might continual learning affect safety and alignment?
LessWrong · 13 iun. 2026