aiqbench analysis
This research provides a geometric characterization of how context alters truth representations within the residual stream activations of Large Language Models. The study reveals that while adding context generally amplifies the separation between true and false vectors, larger models rely on directional shifts to filter irrelevant information compared to smaller models using magnitude changes. These findings offer critical insight into the internal mechanics of LLM reasoning and how contextual grounding modifies fundamental truth encodings in neural networks.
How Context Shapes Truth: Geometric Transformations of Statement-level Truth Representations in LLMs
arXiv:2601.06599v3 Announce Type: replace Abstract: Large Language Models (LLMs) often encode whether a statement is true as a vector in their residual stream activations. These vectors, also known as truth vectors, have been studied in prior work, however how they change when context is introduced remains unexplored. We study this question by measuring (1) the directional change ($\theta$) between the truth vectors with and without context and (2) the relative magnitude of the truth vectors upon adding context. Across four LLMs and four datasets, we find that (1) truth vectors are roughly orthogonal in early layers, converge in middle layers, and may stabilize or continue increasing in later layers; (2) adding context generally increases the truth vector magnitude, i.e., the separation between true and false representations in the activation space is amplified; (3) larger models distinguish relevant from irrelevant context mainly through directional change ($\theta$), while smaller models show this distinction through magnitude differences. We also find that context conflicting with parametric knowledge produces larger geometric changes than parametrically aligned context. To the best of our knowledge, this is the first work that provides a geometric characterization of how context transforms the truth vector in the activation space of LLMs.
Abstract:Large Language Models (LLMs) often encode whether a statement is true as a vector in their residual stream activations. These vectors, also known as truth vectors, have been studied in prior work, however how they change when context is introduced remains unexplored. We study this question by measuring (1) the directional change ($\theta$) between the truth vectors with and without context and (2) the relative magnitude of the truth vectors upon adding context. Across four LLMs and four datasets, we find that (1) truth vectors are roughly orthogonal in early layers, converge in middle layers, and may stabilize or continue increasing in later layers; (2) adding context generally increases the truth vector magnitude, i.e., the separation between true and false representations in the activation space is amplified; (3) larger models distinguish relevant from irrelevant context mainly through directional change ($\theta$), while smaller models show this distinction through magnitude differences. We also find that context conflicting with parametric knowledge produces larger geometric changes than parametrically aligned context. To the best of our knowledge, this is the first work that provides a geometric characterization of how context transforms the truth vector in the activation space of LLMs.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2601.06599 [cs.CL]
(or arXiv:2601.06599v3 [cs.CL] for this version)
Full text under the publisher's open licence · 210 words
Related coverage
How Context Shapes Truth: Geometric Transformations of Statement-level Truth Representations in LLMs
arXiv · Jun 9, 2026
Why do models task game?
AI Alignment Forum · Aug 6, 2026
How do neural networks regress cubic polynomials? Apparently, they use a trick invented in Milan 500 years ago
LessWrong · Jul 22, 2026
Links #5: 2026/07
LessWrong · Jul 31, 2026