analiza aiqbench
This research introduces "Reverse Probing," a novel framework that enables large language models to signal token-level uncertainty specifically within clinical text summarization tasks. By leveraging internal model activations rather than generating new samples, the method significantly improves reliability metrics and reduces computational overhead compared to existing open-domain approaches. This advancement is critical for deploying AI safely in high-stakes medical environments where precise confidence localization is essential.
Reverse Probing: Supervised Token-level Uncertainty Quantification for Large Language Models in Clinical Text
arXiv:2605.28740v2 Announce Type: replace-cross Abstract: As large language models are increasingly deployed for clinical text, ensuring they can reliably signal their own uncertainty becomes critical. Most existing uncertainty quantification (UQ) methods are designed for open-domain generation and cannot localize uncertainty at the token or span level in long clinical text. We propose Reverse Probing, the first UQ framework specialized for clinical summarization, which estimates token-level uncertainty directly from pre-existing labeled summaries. Rather than sampling new outputs, Reverse Probing treats the text as a probe into the model's internal state, extracting uncertainty signals from four categories of internal activations. We evaluate on two expert-annotated clinical datasets and outperform eight adapted baselines on all metrics, achieving up to 4 times higher AUPRC while reducing inference time and computational costs. Feature analysis reveals that delta energy and neighborhood context are the most consistent predictors across all models. This study offers interpretable insights into how models internally respond to unsupported clinical content.
Abstract:As large language models are increasingly deployed for clinical text, ensuring they can reliably signal their own uncertainty becomes critical. Most existing uncertainty quantification (UQ) methods are designed for open-domain generation and cannot localize uncertainty at the token or span level in long clinical text. We propose Reverse Probing, the first UQ framework specialized for clinical summarization, which estimates token-level uncertainty directly from pre-existing labeled summaries. Rather than sampling new outputs, Reverse Probing treats the text as a probe into the model's internal state, extracting uncertainty signals from four categories of internal activations. We evaluate on two expert-annotated clinical datasets and outperform eight adapted baselines on all metrics, achieving up to 4 times higher AUPRC while reducing inference time and computational costs. Feature analysis reveals that delta energy and neighborhood context are the most consistent predictors across all models. This study offers interpretable insights into how models internally respond to unsupported clinical content.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2605.28740 [cs.CL]
(or arXiv:2605.28740v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2605.28740 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 249 cuvinte
Subiecte conexe
TrAC: Trace-Conditioned Answer Consistency for Efficient Uncertainty Quantification in LLMs
arXiv · 5 aug. 2026
Synthesis Superintelligence: from Semiconductors to Superconductors — Periodic Labs’ Liam Fedus and Ekin Dogus Cubuk
Latent Space · 8 oct. 2026
Does DiffusionGemma do latent reasoning?
AI Alignment Forum · 16 aug. 2026
J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses
LessWrong · 8 oct. 2026