aiqbench analysis
This paper applies Gaussian Process Upper Confidence Bound bandits to allocating a shared budget across segments, using a Matérn + Linear kernel selected in a synthetic simulator and then tested in a randomized controlled trial in production. The reported gain is over an ε-greedy baseline, but the abstract gives no effect sizes, so the practical lift is unquantified. For ML readers, the main value is the empirical evidence that GP priors can share information across neighboring configurations under delayed, noisy feedback and limited adjustment frequency.
Gaussian Process Bandits for Constraint Black Box Lever Optimization
arXiv:2610.36935v1 Announce Type: new Abstract: Optimizing discrete allocation weights across competing segments of a shared budget is a fundamental challenge in resource-constrained, sequential decision-making environments. In practice, this optimization is often hindered by delayed, noisy, and aggregate feedback, alongside strict operational constraints on how frequently weights can be adjusted. Because of the resulting combinatorial action space and extreme data sparsity, classical multi-armed bandit algorithms that evaluate arms independently are rendered highly inefficient. To overcome these limitations, we propose a segmentation-agnostic approach utilizing Gaussian Process Upper Confidence Bound (GP-UCB) algorithms. By placing a Gaussian Process prior on the reward function, our method effectively shares information across neighboring configurations, drastically reducing the necessary exploration horizon. In this paper, we bridge the gap between theoretical bandit research and production-scale deployment. We first introduce a high-fidelity synthetic simulator to evaluate various kernel configurations under stationary, drifting, and reactive environments, identifying a composite Mat\'ern + Linear kernel as the most robust choice for overcoming local noise and global shifts. Finally, we validate our findings through a large-scale Randomized Controlled Trial (RCT) in a live production environment. Our empirical results demonstrate that the GP-UCB approach delivers a statistically significant efficiency lift compared to a standard $\varepsilon$-greedy baseline, offering a highly practical solution for real-world black-box lever optimization.
Abstract:Optimizing discrete allocation weights across competing segments of a shared budget is a fundamental challenge in resource-constrained, sequential decision-making environments. In practice, this optimization is often hindered by delayed, noisy, and aggregate feedback, alongside strict operational constraints on how frequently weights can be adjusted. Because of the resulting combinatorial action space and extreme data sparsity, classical multi-armed bandit algorithms that evaluate arms independently are rendered highly inefficient. To overcome these limitations, we propose a segmentation-agnostic approach utilizing Gaussian Process Upper Confidence Bound (GP-UCB) algorithms. By placing a Gaussian Process prior on the reward function, our method effectively shares information across neighboring configurations, drastically reducing the necessary exploration horizon. In this paper, we bridge the gap between theoretical bandit research and production-scale deployment. We first introduce a high-fidelity synthetic simulator to evaluate various kernel configurations under stationary, drifting, and reactive environments, identifying a composite Matérn + Linear kernel as the most robust choice for overcoming local noise and global shifts. Finally, we validate our findings through a large-scale Randomized Controlled Trial (RCT) in a live production environment. Our empirical results demonstrate that the GP-UCB approach delivers a statistically significant efficiency lift compared to a standard $\varepsilon$-greedy baseline, offering a highly practical solution for real-world black-box lever optimization.
7 pages, 3 figures, 1 table. Includes results from a large-scale randomized controlled trial in a production environment
Machine Learning (cs.LG)
arXiv:2610.36935 [cs.LG]
(or arXiv:2610.36935v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.36935 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 318 words
Related coverage
MDPO: Multi-constraint Dexterous Policy Optimization with Reinforcement Learning
arXiv · Oct 27, 2026
Binary Optimization with Complex Constraints via Quantum Approximate Multi-Objective Optimization
arXiv · Sep 15, 2026
STAIF: A Stage-wise Optimization for Complex Instruction Following
arXiv · Jul 28, 2026
ORACLE: Optimizer-Relative Alignment for Constrained LEarning
arXiv · Oct 8, 2026