aiqbench analysis
This paper studies how to detect hallucinations in LLMs by training lightweight probes on internal model signals, and it compares design choices like feature type, prompting, and label construction under matched conditions. Its main caveat is that in-domain accuracy alone is a weak proxy for progress, since structured or compressed features held up better under distribution shift. For readers, the practical takeaway is that probe-based uncertainty estimators should be evaluated on deployment-like shifts, and benchmark-pretrained probes look like a reasonable starting point that still needs target-domain adaptation.
From Signals to Transfer: A Factorised Study of Probe-Based Uncertainty Estimation in Large Language Models
arXiv:2606.27679v2 Announce Type: replace-cross Abstract: Probe-based uncertainty estimation (UE) has emerged as a prominent approach to detect hallucinations in Large Language Models (LLMs) by learning uncertainty from internal model signals. Yet, recent methods vary simultaneously across feature design, training data construction, and evaluation setting, obscuring what actually drives performance. To address this issue, we propose a factorised study of probe-based UE under matched conditions. Our results show that raw hidden states and attention features are difficult to outperform in-domain. However, under distribution shift, structured and compressed features are more robust, suggesting that in-domain performance alone is insufficient to measure progress. Furthermore, prompting and label construction significantly affect probe behaviour. Building on these empirical design principles, we construct benchmark-pretrained probes that retain useful discriminative signal on open-ended factual generation. Their competitive performance relative to limited-label target setting probes and sampling-based estimators suggests that they can provide a practical starting point for subsequent target-domain adaptation. More broadly, our work encourages deployment-oriented evaluation of probe-based uncertainty estimators.
Abstract:Probe-based uncertainty estimation (UE) has emerged as a prominent approach to detect hallucinations in Large Language Models (LLMs) by learning uncertainty from internal model signals. Yet, recent methods vary simultaneously across feature design, training data construction, and evaluation setting, obscuring what actually drives performance. To address this issue, we propose a factorised study of probe-based UE under matched conditions. Our results show that raw hidden states and attention features are difficult to outperform in-domain. However, under distribution shift, structured and compressed features are more robust, suggesting that in-domain performance alone is insufficient to measure progress. Furthermore, prompting and label construction significantly affect probe behaviour. Building on these empirical design principles, we construct benchmark-pretrained probes that retain useful discriminative signal on open-ended factual generation. Their competitive performance relative to limited-label target setting probes and sampling-based estimators suggests that they can provide a practical starting point for subsequent target-domain adaptation. More broadly, our work encourages deployment-oriented evaluation of probe-based uncertainty estimators.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2606.27679 [cs.CL]
(or arXiv:2606.27679v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.27679 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 255 words
Related coverage
When Uncertainty Isn't Enough: An Empirical Study of Self-Correction in Code Generation
arXiv · Aug 19, 2026
A Unified Mechanistic Analysis of Knowledge- and Safety-Based Refusals
arXiv · Sep 2, 2026
What We Learned Trying to Catch AI Liars: An Aletheia's Quest Retrospective
EleutherAI · Aug 25, 2026
Open Distillation of Hereditary Traits
AI Alignment Forum · Jul 14, 2026