analiza aiqbench
This research introduces ICE, a rigorous framework that demonstrates explanation faithfulness in LLMs is not an intrinsic property but varies significantly depending on the intervention method used. By employing statistical testing against randomized baselines across multiple languages and models, the study reveals that many standard attention-based explanations fail to reflect genuine reasoning compared to random chance. This work fundamentally shifts how we validate model interpretability, proving that high accuracy does not guarantee faithful internal logic without operator-specific verification.
ICE: Intervention-Consistent Explanation Evaluation with Statistical Grounding for LLMs
arXiv:2603.18579v2 Announce Type: replace-cross Abstract: Evaluating whether explanations faithfully reflect a model's reasoning remains an open problem. Existing benchmarks use single interventions without statistical testing, making it impossible to distinguish genuine faithfulness from chance-level performance. We show that faithfulness is not a fixed property but an operator-dependent quantity that changes with the intervention method used to measure it. We introduce ICE (Intervention-Consistent Explanation), a framework that evaluates explanations against random baselines of equal size under multiple operators. Evaluating 7 LLMs across 4 tasks with deletion and retrieval infill operators, we find that switching operators crosses the positive-evidence threshold in 18% of configurations (5 of 28 attention comparisons), with gaps reaching 44 percentage points. Randomized baselines detect anti-faithfulness (explanations worse than random) in nearly one-third of English deletion configurations, invisible without random comparisons. These patterns persist across 6 non-English languages and 2 attribution methods. The methodology generalizes to step-level chain-of-thought evaluation, where preliminary results on 3 frontier models suggest that high accuracy does not imply faithful reasoning.
Abstract:Evaluating whether explanations faithfully reflect a model's reasoning remains an open problem. Existing benchmarks use single interventions without statistical testing, making it impossible to distinguish genuine faithfulness from chance-level performance. We show that faithfulness is not a fixed property but an operator-dependent quantity that changes with the intervention method used to measure it. We introduce ICE (Intervention-Consistent Explanation), a framework that evaluates explanations against random baselines of equal size under multiple operators. Evaluating 7 LLMs across 4 tasks with deletion and retrieval infill operators, we find that switching operators crosses the positive-evidence threshold in 18% of configurations (5 of 28 attention comparisons), with gaps reaching 44 percentage points. Randomized baselines detect anti-faithfulness (explanations worse than random) in nearly one-third of English deletion configurations, invisible without random comparisons. These patterns persist across 6 non-English languages and 2 attribution methods. The methodology generalizes to step-level chain-of-thought evaluation, where preliminary results on 3 frontier models suggest that high accuracy does not imply faithful reasoning.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
I.2.7; I.2.6; I.5.2; G.3
arXiv:2603.18579 [cs.CL]
(or arXiv:2603.18579v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2603.18579 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 264 cuvinte
Subiecte conexe
J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses
LessWrong · 8 oct. 2026
Thousand-dimensional structure
AI Alignment Forum · 30 iul. 2026
The basic argument for making AI better at moral philosophy
LessWrong · 14 oct. 2026
Models Can't Remember Their Training. Neither Can You.
LessWrong · 19 iul. 2026