analiza aiqbench
This paper introduces VBO-MI, a gradient-based Bayesian optimization framework that eliminates the computational bottlenecks of Gaussian processes and Bayesian neural networks by using variational mutual information. By treating the objective function as a strict black box, the method reduces floating-point operations significantly while maintaining consistency guarantees across diverse real-world tasks. The approach represents a meaningful advancement in scalable optimization for expensive black-box functions without relying on traditional parametric priors.
Amortized Acquisition Optimization for Bayesian Optimization with Variational Mutual Information
arXiv:2601.08172v2 Announce Type: replace Abstract: Bayesian optimization (BO) of expensive black-box functions is traditionally addressed with Gaussian processes (GPs), which scale cubically with observations, or Bayesian neural networks (BNNs), which incur costly posterior sampling and inner-loop acquisition optimization. We propose VBO-MI (Variational Bayesian Optimization with Mutual Information), a fully gradient-based BO framework that requires no explicit GP prior or fixed parametric posterior family over objective function and treats it as a strict black box. An actor-critic architecture pairs an action-net with a variational critic that estimates information gain, eliminating the acquisition optimization bottleneck and achieving up to $10^{2}\times$ fewer FLOPs than BNN-BO baselines. A lightweight surrogate network further reduces real function queries to one batch per iteration. We establish consistency guarantees and evaluate VBO-MI on synthetic benchmarks (Ackley, Levy, Griewank) and real-world tasks (Rover Trajectory, Lunar Lander, Pest Control), demonstrating competitive or superior performance over the baselines.
Abstract:Bayesian optimization (BO) of expensive black-box functions is traditionally addressed with Gaussian processes (GPs), which scale cubically with observations, or Bayesian neural networks (BNNs), which incur costly posterior sampling and inner-loop acquisition optimization. We propose VBO-MI (Variational Bayesian Optimization with Mutual Information), a fully gradient-based BO framework that requires no explicit GP prior or fixed parametric posterior family over objective function and treats it as a strict black box. An actor-critic architecture pairs an action-net with a variational critic that estimates information gain, eliminating the acquisition optimization bottleneck and achieving up to $10^{2}\times$ fewer FLOPs than BNN-BO baselines. A lightweight surrogate network further reduces real function queries to one batch per iteration. We establish consistency guarantees and evaluate VBO-MI on synthetic benchmarks (Ackley, Levy, Griewank) and real-world tasks (Rover Trajectory, Lunar Lander, Pest Control), demonstrating competitive or superior performance over the baselines.
Accepted at ICML 2026 Workshop on Decision-Making from Offline Datasets to Online Adaptation: Black-Box Optimization to Reinforcement Learning., Seoul, South Korea, 2026
Machine Learning (cs.LG); Information Theory (cs.IT)
arXiv:2601.08172 [cs.LG]
(or arXiv:2601.08172v2 [cs.LG] for this version)
Text integral sub licența deschisă a publicației · 178 cuvinte
Subiecte conexe
A Unified Information-Theoretic Approach to Constrained Multi-Fidelity Multi-Objective Bayesian Optimization
arXiv · 8 oct. 2026
BOBA: Dynamic Bayesian Optimization through Bayesian Active Inference
arXiv · 23 sept. 2026
Direct Regret Optimization in Bayesian Optimization
arXiv · 7 oct. 2026
The Entangled Dimensions of Decision Theory
LessWrong · 30 iul. 2026