aiqbench analysis
This arXiv study tests whether open-weights LLMs can use representations they induce from context, such as novel semantics or patterns, for next-token prediction and a new "adaptive world modeling" task. The authors report that models often fail to deploy these representations even when the information is encoded internally, and that leading closed reasoning models show the same limitation. For readers, the result is a measured reminder that in-context representation learning does not reliably translate into flexible behavior, which bears on how far current LLMs can adapt after deployment.
Language Models Struggle to Use Representations Learned In-Context
arXiv:2602.04212v3 Announce Type: replace-cross Abstract: Though large language models (LLMs) have enabled great success across a wide variety of tasks, they still appear to fall short of one of the loftier goals of artificial intelligence research: creating an artificial system that can adapt its behavior to radically new contexts upon deployment. One important step towards this goal is to create systems that can induce rich representations of data that are seen in-context, and then flexibly deploy these representations to accomplish goals. Recently, Park et al. (2024) demonstrated that current LLMs are indeed capable of inducing such representation from context (i.e., in-context representation learning). The present study investigates whether LLMs can use these representations to complete simple downstream tasks. We first assess whether open-weights LLMs can use in-context representations for next-token prediction, and then probe models using a novel task, adaptive world modeling. In both tasks, we find evidence that open-weights LLMs struggle to deploy representations of novel semantics that are defined in-context, even if they encode these semantics in their latent representations. Furthermore, we assess closed-source, state-of-the-art reasoning models on the adaptive world modeling task, demonstrating that even the most performant LLMs cannot reliably leverage novel patterns presented in-context. Overall, this work seeks to inspire novel methods for encouraging models to not only encode information presented in-context, but to do so in a manner that supports flexible deployment of this information.
Abstract:Though large language models (LLMs) have enabled great success across a wide variety of tasks, they still appear to fall short of one of the loftier goals of artificial intelligence research: creating an artificial system that can adapt its behavior to radically new contexts upon deployment. One important step towards this goal is to create systems that can induce rich representations of data that are seen in-context, and then flexibly deploy these representations to accomplish goals. Recently, Park et al. (2024) demonstrated that current LLMs are indeed capable of inducing such representation from context (i.e., in-context representation learning). The present study investigates whether LLMs can use these representations to complete simple downstream tasks. We first assess whether open-weights LLMs can use in-context representations for next-token prediction, and then probe models using a novel task, adaptive world modeling. In both tasks, we find evidence that open-weights LLMs struggle to deploy representations of novel semantics that are defined in-context, even if they encode these semantics in their latent representations. Furthermore, we assess closed-source, state-of-the-art reasoning models on the adaptive world modeling task, demonstrating that even the most performant LLMs cannot reliably leverage novel patterns presented in-context. Overall, this work seeks to inspire novel methods for encouraging models to not only encode information presented in-context, but to do so in a manner that supports flexible deployment of this information.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2602.04212 [cs.CL]
(or arXiv:2602.04212v3 [cs.CL] for this version)
From: Michael Lepori Jr. [view email] [v1] Wed, 4 Feb 2026 04:55:07 UTC (222 KB) [v2] Fri, 1 May 2026 03:55:21 UTC (1,198 KB) [v3] Thu, 15 Oct 2026 17:06:29 UTC (1,198 KB)
Full text under the publisher's open licence · 274 words
Related coverage
Benchtalks #3: We taught AI everything except how to learn
Snorkel AI Blog · Jun 25, 2026
Latent reasoning architectures would likely undermine CoT, our strongest oversight tool
AI Alignment Forum · Sep 23, 2026
J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses
LessWrong · Oct 8, 2026
Academia is for Ambition — Alex Zhang, MIT
Latent Space · Oct 2, 2026