analiza aiqbench
This research exposes a critical safety gap in clinical LLMs, demonstrating that high accuracy and answer consistency are poor proxies for faithfulness when the model's reasoning relies on spurious correlations rather than causal medical concepts. The authors introduce three new behavioral metrics—ESI, CFS, and PSS—to quantify this disconnect, revealing that models can confidently agree on answers while sharing almost no underlying reasoning logic. For developers of clinical decision support systems, these findings underscore the urgent need to move beyond standard evaluation benchmarks toward rigorous counterfactual testing that validates the actual drivers of model predictions.
Right Answer, Wrong Reason: Accuracy, Consistency, and Consensus Are Misleading Indicators of LLM Faithfulness in Clinical Decision Support
arXiv:2609.32817v1 Announce Type: new Abstract: Clinical Large Language Models (LLMs) achieve strong medical-exam accuracy; however, a correct answer does not guarantee that the explanation names the concepts that actually drove the decision. We introduce three lightweight, directly interpretable metrics for this faithfulness gap: the Explanation Stability Index (ESI), which measures reasoning consistency across repeated queries; the Causal Faithfulness Score (CFS), which tests whether cited concepts drive predictions via concept ablation; and the Perturbation Stability Score (PSS), which measures robustness to semantic-preserving paraphrases. By evaluating six LLMs on 150 MedQA-USMLE questions (900 model-question observations), we found that only 23.3% of the cited clinical concepts were causally necessary. Correct answers had lower CFS than incorrect answers (0.212 vs. 0.398), answer consistency negatively predicted CFS (Spearman r = -0.466), and model pairs could agree on answers while sharing only 8.8% of cited reasoning concepts. These results show that accuracy, consistency, and consensus are incomplete safety signals for clinical decision-making support. The evidence is behavioral rather than mechanistic: concept ablation tests counterfactual sensitivity of outputs, not internal circuits.
Abstract:Clinical Large Language Models (LLMs) achieve strong medical-exam accuracy; however, a correct answer does not guarantee that the explanation names the concepts that actually drove the decision. We introduce three lightweight, directly interpretable metrics for this faithfulness gap: the Explanation Stability Index (ESI), which measures reasoning consistency across repeated queries; the Causal Faithfulness Score (CFS), which tests whether cited concepts drive predictions via concept ablation; and the Perturbation Stability Score (PSS), which measures robustness to semantic-preserving paraphrases. By evaluating six LLMs on 150 MedQA-USMLE questions (900 model-question observations), we found that only 23.3% of the cited clinical concepts were causally necessary. Correct answers had lower CFS than incorrect answers (0.212 vs. 0.398), answer consistency negatively predicted CFS (Spearman r = -0.466), and model pairs could agree on answers while sharing only 8.8% of cited reasoning concepts. These results show that accuracy, consistency, and consensus are incomplete safety signals for clinical decision-making support. The evidence is behavioral rather than mechanistic: concept ablation tests counterfactual sensitivity of outputs, not internal circuits.
Artificial Intelligence (cs.AI)
arXiv:2609.32817 [cs.AI]
(or arXiv:2609.32817v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2609.32817 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 263 cuvinte
Subiecte conexe
State of the Field: AI for Epistemics and Coordination
LessWrong · 2 oct. 2026
What We Learned Trying to Catch AI Liars: An Aletheia's Quest Retrospective
EleutherAI · 25 aug. 2026
Does ChatGPT really have a strong left-wing bias?
LessWrong · 27 iul. 2026
Cracking the Data Science Case Study Interview
Analytics Vidhya · 26 iul. 2026