analiza aiqbench
This research introduces an Etiology-Aware Attention Supervision framework to improve the diagnostic reliability of Large Language Models by aligning their internal attention mechanisms with structured clinical guidelines. By applying parameter-efficient fine-tuning, the method steers models toward clinically relevant evidence without altering base architectures, yielding a significant 15.65% boost in accuracy for acute abdominal conditions. The work addresses critical safety concerns regarding how LLMs utilize diagnostic information internally to ensure trustworthiness in high-stakes medical applications.
Enhancing Trustworthy Clinical Diagnosis Decision-Making in Large Language Models via Etiology-Aware Attention Supervision
arXiv:2508.00285v2 Announce Type: replace Abstract: Objective: Large Language Models (LLMs) have demonstrated strong capabilities in medical text understanding and generation. However, their trustworthiness in diagnosis-oriented medical tasks remains constrained by the lack of structured guidance on how clinically relevant diagnostic evidence is internally attended to and utilized during model learning. Method: We propose an Etiology-Aware Attention Supervision framework that introduces structured etiological information as an external supervisory signal for training large language models. Specifically, we construct Clinical Etiology Schema (CES) derived from authoritative clinical guidelines for three acute abdominal conditions: acute appendicitis, acute pancreatitis, and acute cholecystitis. Based on CES annotations, we develop an Etiology-Aware Head Identification strategy to identify attention heads that consistently align with etiological evidence. Building on this analysis, we design a structure-guided parameter-efficient fine-tuning approach that steers attention distributions toward clinically relevant evidence through an additional supervision loss, without modifying the base model architecture. Result: Experiments conducted on a Consistent Diagnosis Cohort demonstrate that the proposed framework improves average diagnostic accuracy by 15.65% compared with baseline models. Attention-based metrics, including Inference Focus Score and Inference Attention Frequency, show more concentrated attention on etiologically relevant evidence. External evaluation on a Discrepant Diagnosis Cohort further confirms the robustness of diagnostic performance improvements under real-world clinical inconsistencies.
Abstract:Objective: Large Language Models (LLMs) have demonstrated strong capabilities in medical text understanding and generation. However, their trustworthiness in diagnosis-oriented medical tasks remains constrained by the lack of structured guidance on how clinically relevant diagnostic evidence is internally attended to and utilized during model learning. Method: We propose an Etiology-Aware Attention Supervision framework that introduces structured etiological information as an external supervisory signal for training large language models. Specifically, we construct Clinical Etiology Schema (CES) derived from authoritative clinical guidelines for three acute abdominal conditions: acute appendicitis, acute pancreatitis, and acute cholecystitis. Based on CES annotations, we develop an Etiology-Aware Head Identification strategy to identify attention heads that consistently align with etiological evidence. Building on this analysis, we design a structure-guided parameter-efficient fine-tuning approach that steers attention distributions toward clinically relevant evidence through an additional supervision loss, without modifying the base model architecture. Result: Experiments conducted on a Consistent Diagnosis Cohort demonstrate that the proposed framework improves average diagnostic accuracy by 15.65% compared with baseline models. Attention-based metrics, including Inference Focus Score and Inference Attention Frequency, show more concentrated attention on etiologically relevant evidence. External evaluation on a Discrepant Diagnosis Cohort further confirms the robustness of diagnostic performance improvements under real-world clinical inconsistencies.
Computation and Language (cs.CL)
arXiv:2508.00285 [cs.CL]
(or arXiv:2508.00285v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2508.00285 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 296 cuvinte
Subiecte conexe
MedPAIR: Measuring Whether Physicians and AI Agree on What Matters in Medical QA
Snorkel AI Blog · 2 oct. 2026
AI in healthcare: applications and best practices
Databricks · 24 iul. 2026
MedHallTune: An Instruction-Tuning Benchmark for Mitigating Medical Hallucination in Vision-Language Models
arXiv · 31 iul. 2026
Announcing the 2026 Databricks Customer Awards Industry winners
Databricks · 9 iun. 2026