analiza aiqbench
This preprint introduces Prediction of Prediction (PoP), a technique that detects hallucinations in large language models by fusing intermediate hidden representations across layers during a single forward pass. By leveraging internal activation dynamics rather than auxiliary decoding calls, the method achieves 75.5% accuracy on factual-correctness benchmarks while adding less than 1.2% runtime latency. This approach offers a promising path toward reducing memory overhead and improving reliability in high-stakes AI applications without requiring multiple generation passes.
Prediction of Prediction (PoP): Inter-Layer Activation Fusion for Single-Pass Hallucination Detection in Large Language Models
arXiv:2608.27165v1 Announce Type: new Abstract: Autoregressive large language models (LLMs) routinely generate factually incorrect outputs with high decoding confidence, limiting their deployment in high-stakes workflows. Existing output-stage uncertainty metrics can fail when models are overconfident on false assertions, while multi-sample verification pipelines introduce substantial memory and latency overhead. This work evaluates whether internal hidden-state transition dynamics during generation can signal factual errors without auxiliary decoding calls. We introduce Prediction of Prediction (PoP), a mechanism that captures layer-transition uncertainty by fusing intermediate hidden representations across depth during a single forward pass. Evaluated on the TruthfulQA benchmark using autoregressive transformer backbones, PoP achieves an area under the receiver operating characteristic curve (AUROC) of 75.5% for factual-correctness classification. The mechanism operates within the base forward pass, adding less than 1.2% runtime latency and requiring zero additional generation passes. The numerical results are reported from the author-verified experimental implementation and are bounded by the evaluation scope described below.
Abstract:Autoregressive large language models (LLMs) routinely generate factually incorrect outputs with high decoding confidence, limiting their deployment in high-stakes workflows. Existing output-stage uncertainty metrics can fail when models are overconfident on false assertions, while multi-sample verification pipelines introduce substantial memory and latency overhead. This work evaluates whether internal hidden-state transition dynamics during generation can signal factual errors without auxiliary decoding calls. We introduce Prediction of Prediction (PoP), a mechanism that captures layer-transition uncertainty by fusing intermediate hidden representations across depth during a single forward pass. Evaluated on the TruthfulQA benchmark using autoregressive transformer backbones, PoP achieves an area under the receiver operating characteristic curve (AUROC) of 75.5% for factual-correctness classification. The mechanism operates within the base forward pass, adding less than 1.2% runtime latency and requiring zero additional generation passes. The numerical results are reported from the author-verified experimental implementation and are bounded by the evaluation scope described below.
7 pages, 3 figures, 9 tables. Single-author preprint on white-box, single-pass hallucination detection using inter-layer activation divergence, cross-layer fusion, temporal drift, and calibrated risk scoring. Evaluated on TruthfulQA, HaluEval 2.0, and FaithDial with Llama-3, Qwen2.5, and Mistral backbones
Computation and Language (cs.CL)
arXiv:2608.27165 [cs.CL]
(or arXiv:2608.27165v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2608.27165 arXiv-issued DOI via DataCite (pending registration)
Text integral sub licența deschisă a publicației · 206 cuvinte
Subiecte conexe
DI Series: Serving Kimi-K3 MXFP4 Wide-EP Disaggregated on AMD Instinct MI300X and MI325X
AMD · 8 oct. 2026
GEM Training: How Meta Doubled the Efficiency of Its LLM-Scale Ads Foundation Model
Meta Engineering · 3 aug. 2026
Building Blocks for Foundation Model Training and Inference on AWS
Hugging Face · 11 mai 2026
Optimizing ATOM and vLLM-ATOM for High-Interactivity Inference
AMD · 1 sept. 2026