Bidirectional Information Flow (BIF) - A Sample Efficient Hierarchical Gaussian Process for Bayesian Optimization

arXiv:2505.11294v3 Announce Type: replace Abstract: Hierarchical Gaussian Process (H-GP) models divide problems into different subtasks, allowing different components to address each part, making them well-suited for problems with inherent compositional structure. However, existing H-GP frameworks typically employ one-way information sharing - either top-down or bottom-up - which limits sample efficiency and slows convergence. We propose Bidirectional Information Flow (BIF), which establishes continuous two-way communication. BIF retains the modular structure of hierarchical models-the parent conditions its own posterior on child summaries, treating them as structured priors-while introducing top-down feedback to softly decompose environment observations from the parent into sub-responses. This mutual exchange improves sample efficiency, enables robust training, and allows modular reuse of learned subtask models. We prove analytically that the regret of a GP with a learned kernel scales linearly with the mismatch to the true kernel, tightening in the hierarchical case to the sum of child-level errors. Ablation shows that removing the downward pathway collapses child $R^2$ by up to 58%. Across synthetic, neurostimulation, and HPO benchmarks, BIF achieves up to $4\times$ higher parent $R^2$ and $\sim 100\%$ AUC improvement over vanilla GPBO, and outscores all hierarchical state-of-the-art methods on child $R^2$ given the correct acquisition function, while supporting modular child transfer to novel composite tasks.

Abstract:Hierarchical Gaussian Process (H-GP) models divide problems into different subtasks, allowing different components to address each part, making them well-suited for problems with inherent compositional structure. However, existing H-GP frameworks typically employ one-way information sharing - either top-down or bottom-up - which limits sample efficiency and slows convergence. We propose Bidirectional Information Flow (BIF), which establishes continuous two-way communication. BIF retains the modular structure of hierarchical models-the parent conditions its own posterior on child summaries, treating them as structured priors-while introducing top-down feedback to softly decompose environment observations from the parent into sub-responses. This mutual exchange improves sample efficiency, enables robust training, and allows modular reuse of learned subtask models. We prove analytically that the regret of a GP with a learned kernel scales linearly with the mismatch to the true kernel, tightening in the hierarchical case to the sum of child-level errors. Ablation shows that removing the downward pathway collapses child $R^2$ by up to 58%. Across synthetic, neurostimulation, and HPO benchmarks, BIF achieves up to $4\times$ higher parent $R^2$ and $\sim 100\%$ AUC improvement over vanilla GPBO, and outscores all hierarchical state-of-the-art methods on child $R^2$ given the correct acquisition function, while supporting modular child transfer to novel composite tasks.

Machine Learning (cs.LG)

arXiv:2505.11294 [cs.LG]

(or arXiv:2505.11294v3 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2505.11294 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

analiza aiqbench

Bidirectional Information Flow (BIF) - A Sample Efficient Hierarchical Gaussian Process for Bayesian Optimization

Subiecte conexe