analiza aiqbench
This study defines "prolepsis" as a mechanism where small decoder-only transformers make early, irrevocable commitments to decisions via specific attention heads that resist correction by subsequent layers. By replicating findings on open models like Gemma 2 and Llama 3.2 using only consumer-grade hardware, the research clarifies how planning sites function invisibly to standard residual-stream methods but are detectable through CLT-based steering. The work provides critical insights into the internal architecture of reasoning in lightweight AI systems without relying on large-scale infrastructure or proprietary models.
What Is the Minimum Architecture for Prolepsis? Early Irrevocable Commitment Across Tasks in Small Transformers
arXiv:2604.15010v2 Announce Type: replace-cross Abstract: When do transformers commit to a decision, and what prevents them from correcting it? We introduce prolepsis: a transformer commits early, task-specific attention heads sustain the commitment, and no layer corrects it. Replicating Lindsey et al.'s (2025) planning-site finding on open models (Gemma 2 2B, Llama 3.2 1B), we ask five questions. (Q1) Planning is invisible to six residual-stream methods; among those tested, only CLT-based steering succeeds. (Q2) The single-site spike replicates in shape, at the final prompt token (Anthropic's site is the newline; see the Note added). (Q3) Specific attention heads route the decision to the output, filling a gap flagged as invisible to attribution graphs. (Q4) The evidence is consistent with search within at most 16 layers and commitment beyond, a two-model hypothesis. (Q5) Factual recall shows the same motif at a different network depth, with zero overlap between recurring planning heads and the factual top-10. Prolepsis recurs across tasks in the decoder-only models tested: the template is shared, the routing substrates differ. All experiments run on a single consumer GPU (16 GB VRAM).
Abstract:When do transformers commit to a decision, and what prevents them from correcting it? We introduce prolepsis: a transformer commits early, task-specific attention heads sustain the commitment, and no layer corrects it. Replicating Lindsey et al.'s (2025) planning-site finding on open models (Gemma 2 2B, Llama 3.2 1B), we ask five questions. (Q1) Planning is invisible to six residual-stream methods; among those tested, only CLT-based steering succeeds. (Q2) The single-site spike replicates in shape, at the final prompt token (Anthropic's site is the newline; see the Note added). (Q3) Specific attention heads route the decision to the output, filling a gap flagged as invisible to attribution graphs. (Q4) The evidence is consistent with search within at most 16 layers and commitment beyond, a two-model hypothesis. (Q5) Factual recall shows the same motif at a different network depth, with zero overlap between recurring planning heads and the factual top-10. Prolepsis recurs across tasks in the decoder-only models tested: the template is shared, the routing substrates differ. All experiments run on a single consumer GPU (16 GB VRAM).
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
arXiv:2604.15010 [cs.LG]
(or arXiv:2604.15010v2 [cs.LG] for this version)
Text integral sub licența deschisă a publicației · 194 cuvinte
Subiecte conexe
Coherent (COHR) Q4 2026 Earnings Call Transcript
fool.com · 20 aug. 2026
Jeremy O’Brien, The Physicist Who Bet Quantum Computing On A Chip Foundry
Quantum Zeitgeist · 7 aug. 2026
JIJ and ORCA Computing Demonstrate Hybrid Quantum-Classical Energy Optimization
Quantum Computing Report · 12 iun. 2026
Grok Build CLI vs Claude Code: I Tested Both So You Don’t Have To
Analytics Vidhya · 24 iul. 2026