analiza aiqbench
This position paper argues that interpretability methods which identify concepts a robot policy has learned do not necessarily reveal what caused a specific failure. The authors frame post-incident analysis as an actual-causality problem and report a toy example where current methods give conflicting verdicts because backup components obscure the real cause. For readers, the takeaway is a methodological warning about auditing embodied AI systems, not a demonstrated tool; the proposed research tasks are still open.
From Concepts to Actual Causes: Toward Closing the Concept-Malfunction Gap in Robot Foundation Models
arXiv:2611.00155v1 Announce Type: cross Abstract: Post-incident analysis of a robot foundation model poses a singular causal question: which internal components caused this failure, and to what degree? Mechanistic interpretability (MI), however, establishes general causal claims by decoding the concepts a policy has learned, yet knowing which concepts are encoded is insufficient for identifying the actual cause of a specific incident. In this position paper, we analyze this concept-malfunction gap by posing fault isolation in the framework of actual causality. Specifically, we formalize a fault-isolation query as a tuple of intervened internal components, counterfactual values, contingencies, and read-outs, and lay out conditions for it to be well-specified. Through this lens, we identify shortcomings of current MI approaches: they assume fixed contingencies and read out only immediate effects, letting backups obscure actual causes. We demonstrate on a toy example that these shortcomings can lead to conflicting verdicts. Lastly, based on the well-specification conditions, we outline four open research tasks toward closing the gap.
Abstract:Post-incident analysis of a robot foundation model poses a singular causal question: which internal components caused this failure, and to what degree? Mechanistic interpretability (MI), however, establishes general causal claims by decoding the concepts a policy has learned, yet knowing which concepts are encoded is insufficient for identifying the actual cause of a specific incident. In this position paper, we analyze this concept-malfunction gap by posing fault isolation in the framework of actual causality. Specifically, we formalize a fault-isolation query as a tuple of intervened internal components, counterfactual values, contingencies, and read-outs, and lay out conditions for it to be well-specified. Through this lens, we identify shortcomings of current MI approaches: they assume fixed contingencies and read out only immediate effects, letting backups obscure actual causes. We demonstrate on a toy example that these shortcomings can lead to conflicting verdicts. Lastly, based on the well-specification conditions, we outline four open research tasks toward closing the gap.
4 pages, 1 figure, 1 table. Accepted at the Science of Physical AI Safety (SPAIS) Workshop at CoRL 2026
Robotics (cs.RO); Artificial Intelligence (cs.AI)
arXiv:2611.00155 [cs.RO]
(or arXiv:2611.00155v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2611.00155 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 271 cuvinte