A Calibrated Reflection Approach for Enhancing Confidence Estimation in LLMs

Abstract:A critical challenge in deploying Large Language Models (LLMs) is developing reliable mechanisms to estimate their confidence, enabling systems to determine when to trust model outputs versus seek human intervention. We present a Calibrated Reflection approach for enhancing confidence estimation in LLMs, a framework that combines structured reasoning with distance-aware calibration technique. Our approach introduces three key innovations: (1) a Maximum Confidence Selection (MCS) method that comprehensively evaluates confidence across all possible labels, (2) a reflection-based prompting mechanism that enhances reasoning reliability, and (3) a distance-aware calibration technique that accounts for ordinal relationships between labels. We evaluate our framework on diverse datasets, including HelpSteer2, Llama T-REx, and a proprietary conversational dataset, demonstrating its effectiveness across both conversational and fact-based classification tasks. This work contributes to the broader goal of developing reliable and well-calibrated confidence estimation methods for LLMs, enabling informed decisions about model trust and human judgement.

Computation and Language (cs.CL)

arXiv:2609.04539 [cs.CL]

(or arXiv:2609.04539v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2609.04539 arXiv-issued DOI via DataCite (pending registration)

Proceedings of the 5th Workshop on Trustworthy Natural Language Processing (TrustNLP 2025), pages 399-411, Albuquerque, New Mexico. Association for Computational Linguistics, 2025

https://doi.org/10.18653/v1/2025.trustnlp-main.26 DOI(s) linking to related resources