aiqbench analysis
This research introduces the Label-Shift-Adjusted Bayesian Score (LSA score) to address uncertainty quantification failures in conformal prediction when label distributions shift between source and target domains. By deriving a direct correction based on posterior predictive tilting, the method achieves tighter prediction intervals while maintaining robust coverage compared to existing residual-based or standard Bayesian approaches. The technique is validated specifically on molecular property prediction tasks, offering a practical solution for deploying reliable AI models in environments with evolving data distributions.
Split Conformal Prediction with Label-Shift-Adjusted Bayesian Scores
arXiv:2609.12386v1 Announce Type: new Abstract: Conformal prediction provides distribution-free uncertainty quantification under exchangeability. However, this assumption is violated by label shift, where the marginal distribution of labels changes while the conditional distribution of inputs given labels remains stable. Under such shifts, standard conformal procedures no longer maintain their intended coverage behavior. Existing approaches address this via importance weighting. They pair the reweighting with residual-based nonconformity scores that ignore predictive uncertainty. The resulting intervals have uniform width. Bayesian conformal methods produce adaptive intervals by leveraging predictive distributions. They evaluate conformity under the source predictive, which is misaligned with the target domain under label shift. We propose the \emph{Label-Shift-Adjusted Bayesian Score} (LSA score), a nonconformity score derived from a posterior predictive tilting identity. This identity shows that the target predictive is an importance-weighted transformation of the source predictive. We use it to derive a direct correction to the Bayesian score. We evaluate the method on molecular property prediction under controlled label shift. The LSA score consistently yields shorter intervals than residual-based and source-based Bayesian scores. Coverage in the target domain remains comparable. Under stronger shift, all methods incur some coverage loss due to pseudo-label-based density-ratio estimation. The LSA score is defined for any source predictive with a tractable log-density. We instantiate it with Bayesian Ridge Regression, where the correction admits a closed form.
Abstract:Conformal prediction provides distribution-free uncertainty quantification under exchangeability. However, this assumption is violated by label shift, where the marginal distribution of labels changes while the conditional distribution of inputs given labels remains stable. Under such shifts, standard conformal procedures no longer maintain their intended coverage behavior. Existing approaches address this via importance weighting. They pair the reweighting with residual-based nonconformity scores that ignore predictive uncertainty. The resulting intervals have uniform width. Bayesian conformal methods produce adaptive intervals by leveraging predictive distributions. They evaluate conformity under the source predictive, which is misaligned with the target domain under label shift. We propose the \emph{Label-Shift-Adjusted Bayesian Score} (LSA score), a nonconformity score derived from a posterior predictive tilting identity. This identity shows that the target predictive is an importance-weighted transformation of the source predictive. We use it to derive a direct correction to the Bayesian score. We evaluate the method on molecular property prediction under controlled label shift. The LSA score consistently yields shorter intervals than residual-based and source-based Bayesian scores. Coverage in the target domain remains comparable. Under stronger shift, all methods incur some coverage loss due to pseudo-label-based density-ratio estimation. The LSA score is defined for any source predictive with a tractable log-density. We instantiate it with Bayesian Ridge Regression, where the correction admits a closed form.
Machine Learning (cs.LG)
arXiv:2609.12386 [cs.LG]
(or arXiv:2609.12386v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2609.12386 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 310 words
Related coverage
Conformal Bayes under Continuous Label Shift: Sensitivity Analysis and the Limits of Exact Validity
arXiv · Sep 24, 2026
The Label Complexity of Class-Conditional Coverage under Distribution Shift
arXiv · Jul 21, 2026
Conformal Bayes under Label Shift: Post-Hoc Calibration vs. In-Training Adaptation
arXiv · Jun 12, 2026
Agents for production lines: Trusted decisions in real time
Databricks · Jul 29, 2026