aiqbench analysis
This paper tests whether predictive runtime monitors can flag safety-spec violations in frozen reinforcement-learning locomotion policies under distribution shift. Its most useful finding is negative: ensemble disagreement did not reliably track prediction error or violations, which cautions against using it as a safety-uncertainty signal. Predicted robustness ranked HalfCheetah violations well, but it offered little clear gain over simply using current-state robustness, so readers should treat the monitoring gains as modest and benchmark-specific.
Beyond Ensemble Disagreement: Predictive STL Monitoring of RL Locomotion under Distribution Shift
arXiv:2611.02218v1 Announce Type: cross Abstract: RL locomotion policies trained under nominal simulation conditions may encounter distribution shifts that compromise safety. Predictive monitors could anticipate violations, but their world-model forecasts may become unreliable under shift, and ensemble disagreement is not calibrated. We evaluate predictive Signal Temporal Logic (STL) monitoring for frozen Ant-v5 and HalfCheetah-v5 policies using closed-loop world-model rollouts and a three-model ensemble. Across external-force, observation-noise, and mass shifts, predicted robustness ranks HalfCheetah's future STL violations strongly, whereas disagreement does not reliably track observation prediction error or violations. Under severe noise, violations remain common ($64.2\pm24.4\%$ at $K{=}20$ across three PPO seeds), yet current-state robustness without prediction comes within $0.03$ AUROC of predictive robustness. Penalizing predicted robustness by disagreement does not consistently improve ranking. Ant provides complementary per-specification evidence, but its composite target has no positive violations and its any-violation target is single-class in most evaluated seed-condition slices. Overall, predictive robustness provides useful STL-specific ranking but little demonstrated additional ranking value over the current state; these findings caution against treating ensemble disagreement as a reliable indicator of safety uncertainty.
Abstract:RL locomotion policies trained under nominal simulation conditions may encounter distribution shifts that compromise safety. Predictive monitors could anticipate violations, but their world-model forecasts may become unreliable under shift, and ensemble disagreement is not calibrated. We evaluate predictive Signal Temporal Logic (STL) monitoring for frozen Ant-v5 and HalfCheetah-v5 policies using closed-loop world-model rollouts and a three-model ensemble. Across external-force, observation-noise, and mass shifts, predicted robustness ranks HalfCheetah's future STL violations strongly, whereas disagreement does not reliably track observation prediction error or violations. Under severe noise, violations remain common ($64.2\pm24.4\%$ at $K{=}20$ across three PPO seeds), yet current-state robustness without prediction comes within $0.03$ AUROC of predictive robustness. Penalizing predicted robustness by disagreement does not consistently improve ranking. Ant provides complementary per-specification evidence, but its composite target has no positive violations and its any-violation target is single-class in most evaluated seed-condition slices. Overall, predictive robustness provides useful STL-specific ranking but little demonstrated additional ranking value over the current state; these findings caution against treating ensemble disagreement as a reliable indicator of safety uncertainty.
Robotics (cs.RO); Machine Learning (cs.LG)
arXiv:2611.02218 [cs.RO]
(or arXiv:2611.02218v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2611.02218 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 268 words
Related coverage
Interpretive Variation and Discourse Structure in Detecting Enthymemes in Controversial Political Discourse
arXiv · Oct 24, 2026
Disagreement-Regularized Imitation Learning for Image-Based Continuous Control with Gaussian and Beta Policies
arXiv · Oct 1, 2026
Divergence Decoding: Training-Free Capability Fusion
arXiv · Jul 31, 2026
Entropy, Disagreement, and the Limits of Foundation Models in Genomics
arXiv · Jun 10, 2026