aiqbench analysis
This research addresses the critical challenge of maintaining statistical validity in conformal prediction when faced with distribution shifts. By deriving theoretical lower bounds on target coverage and proposing a source-tuned pseudo-calibration algorithm, it offers a robust method to mitigate performance degradation without sacrificing prediction set utility. The work advances machine learning reliability by providing concrete mechanisms for handling covariate shift through domain adaptation techniques.
Coverage Guarantees for Pseudo-Calibrated Conformal Prediction under Distribution Shift
arXiv:2602.14913v3 Announce Type: replace Abstract: Conformal prediction (CP) offers distribution-free marginal coverage guarantees under an exchangeability assumption, but these guarantees can fail if the data distribution shifts. We analyze the use of pseudo-calibration as a tool to counter this performance loss under a bounded label-conditional covariate shift model. Using tools from domain adaptation, we derive a lower bound on target coverage in terms of the source-domain loss of the classifier and a Wasserstein measure of the shift. Using this result, we provide a method to design pseudo-calibrated sets that inflate the conformal threshold by a slack parameter to keep target coverage above a prescribed level. Finally, we propose a source-tuned pseudo-calibration algorithm that interpolates between hard pseudo-labels and randomized labels as a function of classifier uncertainty. Numerical experiments show that our bounds qualitatively track pseudo-calibration behavior and that the source-tuned scheme mitigates coverage degradation under distribution shift while maintaining nontrivial prediction set sizes.
Abstract:Conformal prediction (CP) offers distribution-free marginal coverage guarantees under an exchangeability assumption, but these guarantees can fail if the data distribution shifts. We analyze the use of pseudo-calibration as a tool to counter this performance loss under a bounded label-conditional covariate shift model. Using tools from domain adaptation, we derive a lower bound on target coverage in terms of the source-domain loss of the classifier and a Wasserstein measure of the shift. Using this result, we provide a method to design pseudo-calibrated sets that inflate the conformal threshold by a slack parameter to keep target coverage above a prescribed level. Finally, we propose a source-tuned pseudo-calibration algorithm that interpolates between hard pseudo-labels and randomized labels as a function of classifier uncertainty. Numerical experiments show that our bounds qualitatively track pseudo-calibration behavior and that the source-tuned scheme mitigates coverage degradation under distribution shift while maintaining nontrivial prediction set sizes.
Under review. 6 pages, 2 figures, 1 table
Machine Learning (cs.LG); Image and Video Processing (eess.IV)
arXiv:2602.14913 [cs.LG]
(or arXiv:2602.14913v3 [cs.LG] for this version)
Full text under the publisher's open licence · 172 words
Related coverage
Coverage Guarantees for Pseudo-Calibrated Conformal Prediction under Distribution Shift
arXiv · Jun 12, 2026
Beyond Uncertainty Sets: Leveraging Optimal Transport to Extend Conformal Predictive Distributions to Multivariate Settings
arXiv · Oct 1, 2026
Optimization-based Online Conformal Prediction for Multi-step Forecasting
arXiv · Jun 10, 2026
Beyond Marginal Validity: Finite-Sample Guarantees for Localized Conformal Prediction
arXiv · Aug 7, 2026