aiqbench analysis
This research addresses a critical failure mode in adaptive online conformal prediction where standard methods miscover counterfactual outcomes resulting from prediction-driven actions. By introducing propensity-weighted recursion and doubly robust variants, the authors provide a mathematically rigorous solution that restores coverage guarantees up to information-theoretic lower bounds. The findings are significant for deploying reliable uncertainty quantification in sequential decision-making systems like financial rebalancing and bandit algorithms.
Counterfactual Online Conformal Prediction Under Adaptive Logging
arXiv:2609.30811v1 Announce Type: new Abstract: Online conformal prediction can fail when predictions shape actions and actions determine which outcomes enter calibration. Standard adaptive methods may retain marginal coverage while systematically miscovering the counterfactual outcomes of rarely selected actions. This paper formalizes the failure through counterfactual coverage and introduces Propensity-Weighted Online Conformal Prediction, an inverse-propensity-weighted recursion that debiases calibration. A doubly robust variant further reduces nuisance bias to the product of outcome-model and propensity errors. Under positivity, the resulting coverage rate matches an information-theoretic lower bound up to logarithmic factors. Experiments on synthetic decision tasks, open bandit data, and financial rebalancing show that PW-OCP and DR-OCP improve counterfactual coverage and downstream regret without sacrificing prediction-set sharpness.
Abstract:Online conformal prediction can fail when predictions shape actions and actions determine which outcomes enter calibration. Standard adaptive methods may retain marginal coverage while systematically miscovering the counterfactual outcomes of rarely selected actions. This paper formalizes the failure through counterfactual coverage and introduces Propensity-Weighted Online Conformal Prediction, an inverse-propensity-weighted recursion that debiases calibration. A doubly robust variant further reduces nuisance bias to the product of outcome-model and propensity errors. Under positivity, the resulting coverage rate matches an information-theoretic lower bound up to logarithmic factors. Experiments on synthetic decision tasks, open bandit data, and financial rebalancing show that PW-OCP and DR-OCP improve counterfactual coverage and downstream regret without sacrificing prediction-set sharpness.
Machine Learning (cs.LG)
arXiv:2609.30811 [cs.LG]
(or arXiv:2609.30811v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2609.30811 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 204 words
Related coverage
Dynamic Regime-Aware Conformal Calibration for Reliable Economic Forecast Intervals under Multiple Distribution Shifts
arXiv · Aug 20, 2026
Staggered Integral Online Conformal Prediction for Safe Dynamics Adaptation with Multi-Step Coverage Guarantees
arXiv · Oct 12, 2026
Adaptive Bayesian Online Learning via Expert Aggregation
arXiv · Jul 23, 2026
Confounding-Valid Conformal Inference for Counterfactual KPIs in Wireless Networks
arXiv · Sep 7, 2026