aiqbench analysis
This research advances safe reinforcement learning by developing adaptive probabilistic shields that dynamically update as agents learn unknown transition probabilities in Markov decision processes. By integrating online model learning with safety constraints, the method allows safety guarantees to tighten alongside improved environmental understanding, addressing a key limitation of static shielding approaches. The study empirically validates this paradigm across multiple environments, offering a robust framework for deploying RL agents in uncertain settings without sacrificing safety.
Adaptive Probabilistic Shielding by Learning MDPs for Safe Reinforcement Learning
arXiv:2608.19836v1 Announce Type: new Abstract: Probabilistic shielding is a technique for safe reinforcement learning (RL). Typically, a static observer -- called the shield -- constrains the learning agent's actions to those for which acting safely remains feasible. Traditionally, the shield is computed from the transition probabilities of the underlying Markov decision process (MDP). Thus, this technique is not applicable when the MDP model is not given a priori, which, unfortunately, is the case in typical RL applications. In this paper, we study the problem of computing a shield in the setting where the transition graph of the MDP is known, but the transition probabilities are unknown. Our approach integrates probabilistic shielding with online model learning: as the RL agent explores the environment, we estimate the transition probabilities. From this estimate, we compute a shield. While the shield may be conservative initially, it adapts as the model estimate becomes more precise. Thus, the shield improves in tandem with the RL agent. This paradigm of adaptive probabilistic shielding raises a number of challenges, such as when to recompute the shield and how to balance between exploration and safety during learning. We empirically evaluate multiple variants of this paradigm across several environments.
Authors:Astrid Horn Brorholt (1), Maris F. L. Galesloot (2), Nils Jansen (2) (3), Kim Guldstrand Larsen (1), Christian Schilling (1) ((1) Aalborg University, Aalborg, Denmark (2) Radboud University, Nijmegen, Netherlands (3) Ruhr University Bochum, Bochum, Germany)
Abstract:Probabilistic shielding is a technique for safe reinforcement learning (RL). Typically, a static observer -- called the shield -- constrains the learning agent's actions to those for which acting safely remains feasible. Traditionally, the shield is computed from the transition probabilities of the underlying Markov decision process (MDP). Thus, this technique is not applicable when the MDP model is not given a priori, which, unfortunately, is the case in typical RL applications. In this paper, we study the problem of computing a shield in the setting where the transition graph of the MDP is known, but the transition probabilities are unknown. Our approach integrates probabilistic shielding with online model learning: as the RL agent explores the environment, we estimate the transition probabilities. From this estimate, we compute a shield. While the shield may be conservative initially, it adapts as the model estimate becomes more precise. Thus, the shield improves in tandem with the RL agent. This paradigm of adaptive probabilistic shielding raises a number of challenges, such as when to recompute the shield and how to balance between exploration and safety during learning. We empirically evaluate multiple variants of this paradigm across several environments.
19 pages, 3 figures, 3 tables. To be published in the proceedings of RV 2026
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Logic in Computer Science (cs.LO)
arXiv:2608.19836 [cs.LG]
(or arXiv:2608.19836v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2608.19836 arXiv-issued DOI via DataCite (pending registration)
Full text under the publisher's open licence · 271 words
Related coverage
Adjustment Speed as a Safety Constraint for Nonstationary Reinforcement Learning
arXiv · Jul 27, 2026
Easy-to-Use Shielding for Reinforcement Learning
arXiv · Jun 3, 2026
Beyond Runtime Enforcement: Shield Synthesis as Defensibility Analysis for Adversarial Networks
arXiv · Jun 12, 2026
Exchange Policy Optimization Algorithm for Semi-Infinite Safe Reinforcement Learning
arXiv · Sep 3, 2026