Bayesian Optimization under Uncertainty for Training a Scale Parameter in Stochastic Models

Abstract:Hyperparameter tuning is a challenging problem especially when the system itself involves uncertainty. Due to noisy function evaluations, optimization under uncertainty can be computationally expensive. In this paper, we present a novel Bayesian optimization framework tailored for hyperparameter tuning under uncertainty, with a focus on optimizing a scale- or precision-type parameter in stochastic models. The proposed method employs a statistical surrogate for the underlying random variable, enabling analytical evaluation of the expectation operator. Moreover, we derive a closed-form expression for the optimizer of the random acquisition function, which significantly reduces computational cost per iteration. Numerical experiments demonstrate substantial gains in optimization efficiency: about 40-fold reduction in data usage relative to a Monte Carlo-based one-dimensional optimization scheme, and an order of magnitude improvement relative to a Gaussian process-based Bayesian optimization baseline.

25 pages, 23 figures, 3 tables. Code: this https URL

Machine Learning (cs.LG); Computational Engineering, Finance, and Science (cs.CE); Optimization and Control (math.OC); Machine Learning (stat.ML)

arXiv:2510.06439 [cs.LG]

(or arXiv:2510.06439v2 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2510.06439 Focus to learn more arXiv-issued DOI via DataCite

ASCE-ASME J. Risk Uncertainty Eng. Syst., Part A: Civ. Eng. 12(4): 04026050 (2026)

https://doi.org/10.1061/AJRUA6.RUENG-1854 Focus to learn more DOI(s) linking to related resources

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.