aiqbench analysis
This research addresses critical safety issues in clinical AI by replacing unreliable black-box confidence scores with a Bayesian framework that explicitly incorporates clinician priors and diagnostic uncertainty. By treating the LLM as a stochastic simulator, the method significantly improves error detection in medical text classification while remaining robust against misleading inputs or out-of-distribution cases.
Uncertainty-Aware Calibrated Clinical Text Classification with Large Language Models
arXiv:2509.19375v2 Announce Type: replace-cross Abstract: Large language models are increasingly used for clinical text classification, where overconfident misclassifications can directly affect patient care. Existing black-box uncertainty methods attach a confidence score to a fixed LLM prediction using softmax probabilities, verbalised confidence, prompt agreement, or generation consistency. These signals are often poorly calibrated and offer no mechanism for combining model evidence with prior clinical belief. We instead formulate closed-set clinical classification as likelihood-free posterior inference over diagnostic hypotheses. A prompt-conditioned LLM is treated as a class-conditional stochastic simulator: for each candidate diagnosis it generates synthetic clinical descriptions, which are compared with the observed case in an embedding summary space. Sequential Monte Carlo Approximate Bayesian Computation then yields a posterior over diagnoses from which both the prediction and its uncertainty are derived. We instantiate three variants spanning a cost-fidelity spectrum: multinomial SMC (M-SMC), Dirichlet SMC (D-SMC), and an amortised Dirichlet posterior network (ABI) for deployment-time inference. Across three clinical benchmarks and six open LLMs, the inferred posterior separates correct predictions from errors more sharply than black-box confidence, with misclassification-detection AUROC on MultiCare of 0.923 for M-SMC against 0.760 for softmax and 0.584 for a consistency baseline. The posterior also accepts an explicit clinician prior, recovering the correct diagnosis from a deliberately misleading one, and remains diffuse on out-of-distribution presentations. We release OLB-300, an expert-validated clinical uncertainty-quantification benchmark of oral-lesion vignettes.
Abstract:Large language models are increasingly used for clinical text classification, where overconfident misclassifications can directly affect patient care. Existing black-box uncertainty methods attach a confidence score to a fixed LLM prediction using softmax probabilities, verbalised confidence, prompt agreement, or generation consistency. These signals are often poorly calibrated and offer no mechanism for combining model evidence with prior clinical belief. We instead formulate closed-set clinical classification as likelihood-free posterior inference over diagnostic hypotheses. A prompt-conditioned LLM is treated as a class-conditional stochastic simulator: for each candidate diagnosis it generates synthetic clinical descriptions, which are compared with the observed case in an embedding summary space. Sequential Monte Carlo Approximate Bayesian Computation then yields a posterior over diagnoses from which both the prediction and its uncertainty are derived. We instantiate three variants spanning a cost-fidelity spectrum: multinomial SMC (M-SMC), Dirichlet SMC (D-SMC), and an amortised Dirichlet posterior network (ABI) for deployment-time inference. Across three clinical benchmarks and six open LLMs, the inferred posterior separates correct predictions from errors more sharply than black-box confidence, with misclassification-detection AUROC on MultiCare of 0.923 for M-SMC against 0.760 for softmax and 0.584 for a consistency baseline. The posterior also accepts an explicit clinician prior, recovering the correct diagnosis from a deliberately misleading one, and remains diffuse on out-of-distribution presentations. We release OLB-300, an expert-validated clinical uncertainty-quantification benchmark of oral-lesion vignettes.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
arXiv:2509.19375 [cs.LG]
(or arXiv:2509.19375v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2509.19375 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 321 words
Related coverage
Introducing NV-Reason-CT Open 3D CT VLM for Radiologist Chain-of-Thought Reasoning
NVIDIA · Sep 23, 2026
NVIDIA’s NV-Reason-CT brings AI reasoning to 3D CT scans
Quantum Zeitgeist · Sep 24, 2026
A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined
arXiv · Oct 2, 2026
Uncertainty-Aware Decision Making in Multimodal Large Language Models
arXiv · Aug 19, 2026