aiqbench analysis
This research investigates the reliability of safety certificates in systems using lossy encoders, revealing that a latent model can falsely report safety even when physical states become arbitrarily unsafe if discarded dynamics are undetectable. The study establishes rigorous conditions under which these latent barriers transfer to the real world, distinguishing between cases where safety margins exist and those where they do not. For AI practitioners deploying learned representations in critical control systems, this work highlights a fundamental gap between abstract model confidence and physical reality that cannot be ignored.
Latent Safety Filters: When a Lossy Encoder Admits a Transferable Certificate
arXiv:2610.04297v1 Announce Type: cross Abstract: Latent safety filters certify safety on a learned low-dimensional representation of the state, enabling constraints that resist analytic description. Because the encoder is lossy, a filter can report safe while the physical state is unsafe, with no detectable model error. Existing transfer conditions leave the effect of discarded safety information implicit. We ask when a lossy encoder admits a safety certificate that transfers to the physical system, and show the answer is governed by the detectability of the discarded safety-relevant dynamics. We construct a system whose latent model is exact and whose latent signals always report safe, while the physical state becomes arbitrarily unsafe. For this system no certificate exists and no monitor downstream of the encoder can detect the failure. When the discarded dynamics contract, a latent barrier certifies true safety up to two explicit margins, one for the latent-model error and one for the variation of safety across states the encoder cannot distinguish. In the linear case and under boundedness and non-degeneracy conditions, every calibrated barrier transfers with a finite margin when the safety-relevant subspace is detectable, and none does otherwise. On learned cartpole encoders, the model error does not indicate for which representations the estimated bound is non-vacuous, while the second margin does.
Abstract:Latent safety filters certify safety on a learned low-dimensional representation of the state, enabling constraints that resist analytic description. Because the encoder is lossy, a filter can report safe while the physical state is unsafe, with no detectable model error. Existing transfer conditions leave the effect of discarded safety information implicit. We ask when a lossy encoder admits a safety certificate that transfers to the physical system, and show the answer is governed by the detectability of the discarded safety-relevant dynamics. We construct a system whose latent model is exact and whose latent signals always report safe, while the physical state becomes arbitrarily unsafe. For this system no certificate exists and no monitor downstream of the encoder can detect the failure. When the discarded dynamics contract, a latent barrier certifies true safety up to two explicit margins, one for the latent-model error and one for the variation of safety across states the encoder cannot distinguish. In the linear case and under boundedness and non-degeneracy conditions, every calibrated barrier transfers with a finite margin when the safety-relevant subspace is detectable, and none does otherwise. On learned cartpole encoders, the model error does not indicate for which representations the estimated bound is non-vacuous, while the second margin does.
Systems and Control (eess.SY); Machine Learning (cs.LG); Robotics (cs.RO)
arXiv:2610.04297 [eess.SY]
(or arXiv:2610.04297v1 [eess.SY] for this version)
https://doi.org/10.48550/arXiv.2610.04297 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 306 words
Related coverage
J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses
LessWrong · Oct 8, 2026

PIRAMID: Progress and Plans
LessWrong · Jul 27, 2026

Reflections on unlearning and inoculation
LessWrong · Sep 20, 2026
How to Train Your Latent Control Barrier Function: Smooth Safety Filtering Under Hard-to-Model Constraints
arXiv · Oct 14, 2026