analiza aiqbench
This paper introduces a statistical-learning framework designed to analyze optimization algorithms by treating their trajectories as probability distributions, enabling population-level performance studies and data-driven learning of these methods. By integrating PAC-Bayesian generalization guarantees with experiments across convex and non-convex problems, the work advances fundamental machine learning theory regarding how optimizers behave under uncertainty.
A Probabilistic Framework for Learnable Optimization Algorithms
arXiv:2408.11629v2 Announce Type: replace Abstract: We propose a statistical-learning framework for optimization algorithms. The framework is based on probability distributions over optimization trajectories induced by a distribution of optimization problems and a learnable optimization algorithm. Within this setting, optimization performance is represented through measurable performance functionals, including stopping times, contraction factors, and trajectory-level properties. The resulting framework provides a statistical perspective on optimization algorithms, allows their performance to be studied at the population level, and naturally accommodates the learning of optimization algorithms from data. Moreover, it enables the application of statistical-learning techniques to optimization, including PAC-Bayesian generalization guarantees for bounded optimization-specific performance measures. Experiments on convex, non-smooth, non-convex, and stochastic optimization problems illustrate the framework and demonstrate the statistical nature of optimization performance.
Abstract:We propose a statistical-learning framework for optimization algorithms. The framework is based on probability distributions over optimization trajectories induced by a distribution of optimization problems and a learnable optimization algorithm. Within this setting, optimization performance is represented through measurable performance functionals, including stopping times, contraction factors, and trajectory-level properties. The resulting framework provides a statistical perspective on optimization algorithms, allows their performance to be studied at the population level, and naturally accommodates the learning of optimization algorithms from data. Moreover, it enables the application of statistical-learning techniques to optimization, including PAC-Bayesian generalization guarantees for bounded optimization-specific performance measures. Experiments on convex, non-smooth, non-convex, and stochastic optimization problems illustrate the framework and demonstrate the statistical nature of optimization performance.
Machine Learning (cs.LG); Probability (math.PR)
arXiv:2408.11629 [cs.LG]
(or arXiv:2408.11629v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2408.11629 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 212 cuvinte
Subiecte conexe
Probabilistic Multi-Source Data Selection via Learnable Categorical Priors: Variance Reduction and Guaranteed Derandomization
arXiv · 15 oct. 2026
Exploding variance of means of exponentials: least-squares to the rescue – Machine Learning Research Blog
francisbach.com · 27 sept. 2026
Rethinking Learnability in Offline Data-driven Optimization
arXiv · 2 sept. 2026
Hyper Algorithm Design Agent: Evolving Learnable Optimizer from Zero
arXiv · 29 sept. 2026