aiqbench analysis
This research introduces a framework to transform mechanistic interpretability findings into mathematically verifiable claims, moving beyond qualitative ablation studies at GPT-2 scale. By replacing attention heads with synthesized programs and freezing most parameters, the authors achieve solver-checkable proofs of functional equivalence and robustness for specific task circuits. This approach establishes a rigorous standard for validating AI model internals through formal verification rather than empirical observation alone.
Towards Verifiable Transformers: Solver-Checkable Circuit Explanations
arXiv:2605.24033v2 Announce Type: replace Abstract: Mechanistic interpretability typically discovers circuits and then argues what they do from examples and ablations. We introduce Verifiable Transformers, a framework for turning task-localized circuits into bounded, solver-checkable claims: projected functional equivalence, task-relevant invariance, edge necessity, and robustness to continuous final-residual perturbations. At small scale, we directly verify all four properties for quote-closing and bracket-type circuits, including program-mediated circuits whose attention selection is entirely symbolic. At GPT-2 scale, we remove LayerNorm from a sparsemax/LeakyReLU model after training with a +0.0087 OpenWebText loss increase, replace retained attention heads with synthesized restricted-DSL programs, and calibrate only program-local readouts while freezing and hashing every other parameter. The resulting three-edge quote circuit (embedding $\to$ MLP 0 $\to$ program head $\to$ logits) verifies all four properties over a hash-pinned 1,280-prompt domain in linear real arithmetic: 1,280/1,280 equivalence and invariance, 640 edge-necessity witnesses per edge, and robustness at $\epsilon = 0.01$ with minimum certified radius 0.01515. For two alphabet variants of the same opener-detection/copy-type task, untouched gates expose a localization frontier: bracket-type extraction is exact only with all 144 heads, while the constructed quote artifact verifies with three edges. Naive discovery-based verification failed for measured reasons; at scale, we find we must build the object we can verify. The verified object is a calibrated artifact, not the unmodified model, and all claims are bounded to declared domains.
Abstract:Mechanistic interpretability typically discovers circuits and then argues what they do from examples and ablations. We introduce Verifiable Transformers, a framework for turning task-localized circuits into bounded, solver-checkable claims: projected functional equivalence, task-relevant invariance, edge necessity, and robustness to continuous final-residual perturbations. At small scale, we directly verify all four properties for quote-closing and bracket-type circuits, including program-mediated circuits whose attention selection is entirely symbolic. At GPT-2 scale, we remove LayerNorm from a sparsemax/LeakyReLU model after training with a +0.0087 OpenWebText loss increase, replace retained attention heads with synthesized restricted-DSL programs, and calibrate only program-local readouts while freezing and hashing every other parameter. The resulting three-edge quote circuit (embedding $\to$ MLP 0 $\to$ program head $\to$ logits) verifies all four properties over a hash-pinned 1,280-prompt domain in linear real arithmetic: 1,280/1,280 equivalence and invariance, 640 edge-necessity witnesses per edge, and robustness at $\epsilon = 0.01$ with minimum certified radius 0.01515. For two alphabet variants of the same opener-detection/copy-type task, untouched gates expose a localization frontier: bracket-type extraction is exact only with all 144 heads, while the constructed quote artifact verifies with three edges. Naive discovery-based verification failed for measured reasons; at scale, we find we must build the object we can verify. The verified object is a calibrated artifact, not the unmodified model, and all claims are bounded to declared domains.
23 pages. v2: adds GPT-2-scale verified distillation (three-edge verified quote circuit), LayerNorm removal for sparsemax models, and gated localization protocols
Machine Learning (cs.LG); Logic in Computer Science (cs.LO)
arXiv:2605.24033 [cs.LG]
(or arXiv:2605.24033v2 [cs.LG] for this version)
Full text under the publisher's open licence · 257 words
Related coverage
News
Quantum Computing Report · Jul 25, 2026
Performance Profiling on AMD GPUs – Part 5: Profiling-Driven Kernel Optimization with an AI Code-Assist Tool
AMD · Jul 16, 2026
Value Leakage: An LLM’s Answers Are Silently Shaped by Its Own Values
AI Alignment Forum · Jul 31, 2026
News
Quantum Computing Report · Jun 7, 2026