aiqbench analysis
This research introduces TRACED, a novel framework that evaluates Large Language Model reasoning by analyzing the geometric trajectory of token generation rather than relying solely on scalar probabilities. By mapping cognitive processes to concepts like displacement and curvature, the study distinguishes between valid deduction (stable progress) and hallucinations (unstable fluctuations), offering a robust method for assessing model reliability across diverse benchmarks.
Beyond Scalars: Evaluating and Understanding LLM Reasoning via Geometric Progress and Stability
arXiv:2603.10384v3 Announce Type: replace Abstract: Evaluating LLM reliability via scalar probabilities often fails to capture the structural dynamics of reasoning. We introduce TRACED, a framework that assesses reasoning quality through theoretically grounded geometric kinematics. By decomposing reasoning traces into Progress (displacement) and Stability (curvature), we reveal a distinct topological divergence: correct reasoning manifests as high-progress, stable trajectories, whereas hallucinations are characterized by low-progress, unstable patterns (stalled displacement with high curvature fluctuations). Leveraging these signatures, our probabilistic framework achieves competitive performance and superior robustness across diverse benchmarks. Crucially, TRACED bridges geometry and cognition by mapping high curvature to ''Hesitation Loops'' and displacement to ''Certainty Accumulation'', offering a physical lens to decode the internal dynamics of machine thought.
Abstract:Evaluating LLM reliability via scalar probabilities often fails to capture the structural dynamics of reasoning. We introduce TRACED, a framework that assesses reasoning quality through theoretically grounded geometric kinematics. By decomposing reasoning traces into Progress (displacement) and Stability (curvature), we reveal a distinct topological divergence: correct reasoning manifests as high-progress, stable trajectories, whereas hallucinations are characterized by low-progress, unstable patterns (stalled displacement with high curvature fluctuations). Leveraging these signatures, our probabilistic framework achieves competitive performance and superior robustness across diverse benchmarks. Crucially, TRACED bridges geometry and cognition by mapping high curvature to ''Hesitation Loops'' and displacement to ''Certainty Accumulation'', offering a physical lens to decode the internal dynamics of machine thought.
Artificial Intelligence (cs.AI)
arXiv:2603.10384 [cs.AI]
(or arXiv:2603.10384v3 [cs.AI] for this version)
Full text under the publisher's open licence · 122 words
Related coverage
Social Caption: Evaluating Social Understanding in Multimodal Models
arXiv · Jun 3, 2026
OmniCompass: A Holistic Benchmark for Omni-Modal Video Grounding and Understanding
arXiv · Oct 14, 2026
KoALa-Bench: Evaluating Large Audio Language Models on Korean Speech Understanding and Faithfulness
arXiv · Sep 1, 2026
PhyCheck: Fine-Grained Evidence-Grounded Dataset for Physical Law Understanding in Video-LLMs
arXiv · Aug 4, 2026