analiza aiqbench
This arXiv study demonstrates that linear representations within language models can reliably distinguish between modal categories like possibility and impossibility, challenging earlier skepticism about these capabilities. By correlating these internal vectors with human judgments, the research offers mechanistic insights into how LMs structure knowledge of event plausibility and potentially mirrors human cognitive processes. This work advances our understanding of model interpretability and the fundamental reasoning structures underlying generative AI.
Is This Just Fantasy? Language Model Representations Reflect Human Judgments of Event Plausibility
arXiv:2507.12553v4 Announce Type: replace Abstract: Language models (LMs) are used for a diverse range of tasks, from question answering to writing fantastical stories. In order to reliably accomplish these tasks, LMs must be able to discern the modal category of a sentence (i.e., whether it describes something that is possible, impossible, completely nonsensical, etc.). However, recent studies have called into question the ability of LMs to categorize sentences according to modality (Michaelov et al., 2025; Kauf et al., 2023). In this work, we identify linear representations that discriminate between modal categories within a variety of LMs, or modal difference vectors. Analysis of modal difference vectors reveals that LMs have access to more reliable modal categorization judgments than previously reported. Furthermore, we find that modal difference vectors emerge in a consistent order as models become more competent (i.e., through training steps, layers, and parameter count). Notably, we find that modal difference vectors identified within LM activations can be used to model fine-grained human categorization behavior. This potentially provides a novel view into how human participants distinguish between modal categories, which we explore by correlating projections along modal difference vectors with human participants' ratings of interpretable features. In summary, we derive new insights into LM modal categorization using techniques from mechanistic interpretability, with the potential to inform our understanding of modal categorization in humans.
Abstract:Language models (LMs) are used for a diverse range of tasks, from question answering to writing fantastical stories. In order to reliably accomplish these tasks, LMs must be able to discern the modal category of a sentence (i.e., whether it describes something that is possible, impossible, completely nonsensical, etc.). However, recent studies have called into question the ability of LMs to categorize sentences according to modality (Michaelov et al., 2025; Kauf et al., 2023). In this work, we identify linear representations that discriminate between modal categories within a variety of LMs, or modal difference vectors. Analysis of modal difference vectors reveals that LMs have access to more reliable modal categorization judgments than previously reported. Furthermore, we find that modal difference vectors emerge in a consistent order as models become more competent (i.e., through training steps, layers, and parameter count). Notably, we find that modal difference vectors identified within LM activations can be used to model fine-grained human categorization behavior. This potentially provides a novel view into how human participants distinguish between modal categories, which we explore by correlating projections along modal difference vectors with human participants' ratings of interpretable features. In summary, we derive new insights into LM modal categorization using techniques from mechanistic interpretability, with the potential to inform our understanding of modal categorization in humans.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2507.12553 [cs.CL]
(or arXiv:2507.12553v4 [cs.CL] for this version)
From: Michael Lepori Jr. [view email] [v1] Wed, 16 Jul 2025 18:04:26 UTC (659 KB) [v2] Wed, 25 Feb 2026 23:03:52 UTC (675 KB) [v3] Mon, 27 Apr 2026 19:53:17 UTC (681 KB) [v4] Tue, 13 Oct 2026 21:05:35 UTC (681 KB)
Text integral sub licența deschisă a publicației · 274 cuvinte
Subiecte conexe
The J-Space Debate, Agent Swarms, and Pacing Frontier AI - Digital Minds Newsletter #4
LessWrong · 18 sept. 2026
Models Can't Remember Their Training. Neither Can You.
LessWrong · 19 iul. 2026
[AINews] High Taste Testers Ep. 1
Latent Space · 17 oct. 2026
Deconfusing Corrigibility
AI Alignment Forum · 17 oct. 2026