aiqbench analysis
This research introduces a distributionally robust framework for learning hyperparameters in convex optimization, effectively bridging the gap between classical learning-to-optimize methods and worst-case performance estimation. By minimizing a Wasserstein-robust version of the performance estimation problem, the approach yields algorithms with certifiable out-of-sample guarantees that outperform both vanilla baselines and standard worst-case optimal designs. The methodology demonstrates significant practical utility on tasks ranging from logistic regression to linear programming by leveraging stochastic gradient descent over semidefinite programs.
Learning Distributionally Robust First-Order Methods for Convex Optimization
arXiv:2605.06585v2 Announce Type: replace Abstract: We propose a distributionally robust approach to learning hyperparameters (e.g., step sizes and momentum coefficients) for first-order methods in convex optimization. Given a dataset of problem instances, we minimize a Wasserstein distributionally robust version of the performance estimation problem (PEP). Our framework unifies two extremes. As the robustness radius vanishes, we recover classical learning to optimize (L2O), and as it grows we recover the PEP worst-case algorithm design problem. We solve the resulting problem with stochastic gradient descent, differentiating through the solution of an inner semidefinite program at each step. We prove high-probability bounds showing that the true risk of the learned algorithm is at most the in-sample L2O optimum plus a slack that shrinks with the sample size, and is no worse than the worst-case PEP bound. On logistic regression, LASSO, and linear programming examples, our learned algorithms achieve strong out-of-sample performance with certifiable robustness, outperforming both worst-case optimal and vanilla L2O baselines.
Abstract:We propose a distributionally robust approach to learning hyperparameters (e.g., step sizes and momentum coefficients) for first-order methods in convex optimization. Given a dataset of problem instances, we minimize a Wasserstein distributionally robust version of the performance estimation problem (PEP). Our framework unifies two extremes. As the robustness radius vanishes, we recover classical learning to optimize (L2O), and as it grows we recover the PEP worst-case algorithm design problem. We solve the resulting problem with stochastic gradient descent, differentiating through the solution of an inner semidefinite program at each step. We prove high-probability bounds showing that the true risk of the learned algorithm is at most the in-sample L2O optimum plus a slack that shrinks with the sample size, and is no worse than the worst-case PEP bound. On logistic regression, LASSO, and linear programming examples, our learned algorithms achieve strong out-of-sample performance with certifiable robustness, outperforming both worst-case optimal and vanilla L2O baselines.
Machine Learning (cs.LG); Optimization and Control (math.OC)
arXiv:2605.06585 [cs.LG]
(or arXiv:2605.06585v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2605.06585 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 250 words
Related coverage
News
Quantum Computing Report · Jun 7, 2026
First-order Constrained Trilevel Optimization Over Distributed Networks for Robust Coreset Selection
arXiv · Jul 31, 2026
PIRAMID: Progress and Plans
LessWrong · Jul 27, 2026
Latent reasoning architectures would likely undermine CoT, our strongest oversight tool
AI Alignment Forum · Sep 23, 2026