aiqbench analysis
This research advances mechanistic interpretability by using constrained polynomial-zonotope propagation to certify transformer circuit behaviors over bounded input neighborhoods rather than relying on single-input observations. By formulating attention queries as tractable programs that preserve mathematical identities like the softmax simplex, the framework provides rigorous guarantees where empirical heuristics often fail. This methodological shift is critical for validating internal mechanisms in safety-critical AI systems and refining our understanding of model robustness under perturbation.
Certified Mechanistic Interpretability: Lifting Single-Input Findings to Bounded Neighbourhoods
arXiv:2609.26112v1 Announce Type: cross Abstract: Mechanistic interpretability reverse-engineers transformer circuits one input at a time, leaving observed mechanisms without guarantees over bounded input neighbourhoods. We address this gap with a framework based on constrained polynomial-zonotope (CPZ) propagation that lifts mechanistic-interpretability observations from a single input to certified statements over a bounded set of perturbations. Three internal-attention queries (top-$k$ stability, evidence mass, and attention entropy) are formulated as tractable programs over the simplex of attention weights, and CPZ propagation through transformer blocks is shown to preserve the softmax simplex and the LayerNorm zero-mean identity exactly. A recursive Jacobian zonotope construction extends the same certificates across layer depth by linearising the block stack at the input and avoids per-layer generator growth. We instantiate the framework on transformer attention; the resulting certificates offer a way to sharpen mechanistic statements that single-input inspection cannot resolve on its own, and to inform downstream decisions in regimes where empirical heuristics may be misleading.
Abstract:Mechanistic interpretability reverse-engineers transformer circuits one input at a time, leaving observed mechanisms without guarantees over bounded input neighbourhoods. We address this gap with a framework based on constrained polynomial-zonotope (CPZ) propagation that lifts mechanistic-interpretability observations from a single input to certified statements over a bounded set of perturbations. Three internal-attention queries (top-$k$ stability, evidence mass, and attention entropy) are formulated as tractable programs over the simplex of attention weights, and CPZ propagation through transformer blocks is shown to preserve the softmax simplex and the LayerNorm zero-mean identity exactly. A recursive Jacobian zonotope construction extends the same certificates across layer depth by linearising the block stack at the input and avoids per-layer generator growth. We instantiate the framework on transformer attention; the resulting certificates offer a way to sharpen mechanistic statements that single-input inspection cannot resolve on its own, and to inform downstream decisions in regimes where empirical heuristics may be misleading.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2609.26112 [cs.LG]
(or arXiv:2609.26112v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2609.26112 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 249 words
Related coverage
[AINews] Agents disprove Polysemanticity; greatly improve Interpretability & Alignment
Latent Space · Oct 16, 2026
[AINews] High Taste Testers Ep. 1
Latent Space · Oct 17, 2026
Generalization and infinite width
LessWrong · Aug 1, 2026
Increasing Skill Level Recruits Deeper Attention Layers in a Frozen Chess Transformer
LessWrong · Sep 24, 2026