aiqbench analysis
This paper advances causal inference in sequential decision-making by formally separating latent confounding from inherent stochasticity within Markov Decision Processes. By introducing nondeterministic causal models, it enables the identification of robust counterfactual policies that remain valid despite irreducible randomness, validated here on a sepsis treatment simulator with hidden diabetic comorbidities. The work addresses a critical gap in current reinforcement learning frameworks where standard deterministic assumptions fail to capture real-world system noise effectively.
Robust Counterfactual Policy Optimisation via Nondeterministic Causal Models
arXiv:2608.02893v1 Announce Type: new Abstract: Counterfactual inference approaches for sequential decision-making typically assume deterministic causal models, where all randomness stems from latent variables. However, Markov Decision Processes (MDPs) are inherently stochastic. We address this by formalising counterfactual policy optimisation under probabilistic nondeterministic causal models, which properly separates latent confounding from irreducible stochasticity, and here propose a first practical optimisation problem for identifying robust counterfactual policies under a sensitivity analysis framework. We validate our approach on a sepsis treatment simulator, where diabetes status acts as a hidden global confounder.
Abstract:Counterfactual inference approaches for sequential decision-making typically assume deterministic causal models, where all randomness stems from latent variables. However, Markov Decision Processes (MDPs) are inherently stochastic. We address this by formalising counterfactual policy optimisation under probabilistic nondeterministic causal models, which properly separates latent confounding from irreducible stochasticity, and here propose a first practical optimisation problem for identifying robust counterfactual policies under a sensitivity analysis framework. We validate our approach on a sepsis treatment simulator, where diabetes status acts as a hidden global confounder.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2608.02893 [cs.LG]
(or arXiv:2608.02893v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2608.02893 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 180 words
Related coverage
When are safe Pareto improvements individually rational?
LessWrong · Oct 13, 2026
Deep-learning Causal Retrieval Optimization for Efficient e-commerce Distribution in Pinterest
arXiv · Jul 22, 2026
The current bottleneck is political will, not research
AI Alignment Forum · Jul 11, 2026
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · Sep 25, 2026