analiza aiqbench
This research establishes theoretical conditions under which computational approximations of Bayesian posteriors preserve the fast regret guarantees of exact Bayes prediction in online learning settings. By leveraging metrics like Wasserstein distance, the study demonstrates that methods such as projected Langevin algorithms and sparse variational inference can achieve optimal predictive performance with significantly reduced memory and update costs. The work provides a rigorous framework for deploying scalable Bayesian models in resource-constrained environments without sacrificing statistical efficiency.
Fast rates in Bayesian online learning with approximate posteriors
arXiv:2608.25706v1 Announce Type: cross Abstract: Exact Bayes prediction enjoys fast predictive regret guarantees, but exact posterior updating or representation may be too costly for online use. We study when these statistical guarantees are preserved by computational approximations. We show that the cumulative price of posterior approximation can be governed by the interaction between the contraction radius of the exact Gibbs posterior and the Wasserstein distance between the approximate and exact posteriors. Our general theorem shows that whenever exact Bayes prediction achieves a fast regret bound, any approximate posterior method that tracks the exact posterior with sufficient accuracy inherits the same fast regret, up to an additive term determined by the approximation error. Three online learning examples are developed. For linear models with strongly convex regularized losses, a projected Langevin algorithm yields an approximate posterior that achieves logarithmic regret. For an infinite-dimensional canonical exponential family sequence model over a Sobolev ellipsoid, a prior-preserving truncation method attains the minimax predictive regret rate with sublinear memory and constant update cost per observation. For random-design Gaussian process (GP) regression, a sparse variational posterior with inducing variables achieves the same predictive regret rate as the exact GP, but at substantially lower computational cost.
Abstract:Exact Bayes prediction enjoys fast predictive regret guarantees, but exact posterior updating or representation may be too costly for online use. We study when these statistical guarantees are preserved by computational approximations. We show that the cumulative price of posterior approximation can be governed by the interaction between the contraction radius of the exact Gibbs posterior and the Wasserstein distance between the approximate and exact posteriors. Our general theorem shows that whenever exact Bayes prediction achieves a fast regret bound, any approximate posterior method that tracks the exact posterior with sufficient accuracy inherits the same fast regret, up to an additive term determined by the approximation error. Three online learning examples are developed. For linear models with strongly convex regularized losses, a projected Langevin algorithm yields an approximate posterior that achieves logarithmic regret. For an infinite-dimensional canonical exponential family sequence model over a Sobolev ellipsoid, a prior-preserving truncation method attains the minimax predictive regret rate with sublinear memory and constant update cost per observation. For random-design Gaussian process (GP) regression, a sparse variational posterior with inducing variables achieves the same predictive regret rate as the exact GP, but at substantially lower computational cost.
Machine Learning (stat.ML); Machine Learning (cs.LG)
arXiv:2608.25706 [stat.ML]
(or arXiv:2608.25706v1 [stat.ML] for this version)
https://doi.org/10.48550/arXiv.2608.25706 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 290 cuvinte
Subiecte conexe
The Alignment Problem: Computer Science Meets Analytic Philosophy
LessWrong · 18 oct. 2026
Fisher Information based Stochastic Gradient Ascent for Online Learning of Dirichlet Process Mixture and Theory
arXiv · 28 iul. 2026
Fully sequential Gaussian process regression for preference learning
arXiv · 15 oct. 2026
Flow Matching for Fast Posterior Sampling in Bayesian Inverse Problems
arXiv · 5 oct. 2026