aiqbench analysis
This research introduces property-driven causal abstractions for Markov Decision Processes (MDPs) to address the exponential state-space blowup inherent in complex decision-making models. By leveraging causal relations over state variable predicates, the method effectively reduces model complexity while retaining key characteristics necessary for computing near-optimal policies on large-scale benchmarks. The approach demonstrates significant potential for improving scalability and generalization across various MDP types without sacrificing solution quality.
Property-driven Causal Abstractions for Markov Decision Processes
arXiv:2607.26787v1 Announce Type: new Abstract: Markov Decision Processes (MDPs) are widely used as decision-making models, commonly specified over factored state spaces through state variables and their valuations. The exponential blowup in the number of states renders many reasoning tasks in MDPs challenging. Abstractions are promising techniques to reduce MDPs and thus mitigate scalability issues. In this work, we introduce a notion of causality on factored MDPs and a novel property-driven causal abstraction technique that retains many characteristics of the original MDP model. For this, we rely on causal relations over state variable predicates and identify those states that share the same reasons for fulfilling or violating a given abstraction property. We theoretically and empirically compare various causal MDP abstractions using different model types such as MDPs, interval MDPs, or stochastic games. Our evaluation demonstrates the potential of our approach: For several standard benchmarks, we obtain small abstractions that allow us to compute near-optimal policies for the original MDP. Furthermore, our causal abstractions often generalize to related large-scale MDP models.
Abstract:Markov Decision Processes (MDPs) are widely used as decision-making models, commonly specified over factored state spaces through state variables and their valuations. The exponential blowup in the number of states renders many reasoning tasks in MDPs challenging. Abstractions are promising techniques to reduce MDPs and thus mitigate scalability issues. In this work, we introduce a notion of causality on factored MDPs and a novel property-driven causal abstraction technique that retains many characteristics of the original MDP model. For this, we rely on causal relations over state variable predicates and identify those states that share the same reasons for fulfilling or violating a given abstraction property. We theoretically and empirically compare various causal MDP abstractions using different model types such as MDPs, interval MDPs, or stochastic games. Our evaluation demonstrates the potential of our approach: For several standard benchmarks, we obtain small abstractions that allow us to compute near-optimal policies for the original MDP. Furthermore, our causal abstractions often generalize to related large-scale MDP models.
Artificial Intelligence (cs.AI); Logic in Computer Science (cs.LO)
arXiv:2607.26787 [cs.AI]
(or arXiv:2607.26787v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2607.26787 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 263 words
Related coverage
A Unified Causal-Origin Taxonomy of Distributional Shifts in Reinforcement Learning
arXiv · Jun 16, 2026
Blessing from Human-AI Interaction: Super Reinforcement Learning in Confounded Environments
arXiv · Jun 5, 2026
Synthesis Superintelligence: from Semiconductors to Superconductors — Periodic Labs’ Liam Fedus and Ekin Dogus Cubuk
Latent Space · Oct 8, 2026
The Entangled Dimensions of Decision Theory
LessWrong · Jul 30, 2026