A Quantitative Study of Sustained Focus in Large Language Models via Repetitive Deterministic Prediction Tasks

Abstract:We investigate the performance of large language models (LLMs) on repetitive deterministic prediction tasks and study how the sequence accuracy rate (SAR) scales with output length. Each such task involves the repetition of the same operation $N$ times. Examples of such tasks include letter replacement in letter strings following a given rule, integer addition, and multiplication of string operators in many-body quantum mechanics. If the LLM performs the task by a simple repetition algorithm, the success rate would follow an exponential decay with sequence length. In contrast, our experiments on leading LLMs reveal a crossover that is sharper than exponential: $-\log\mathrm{SAR}$ grows super-linearly with $N$, and accuracy collapses around a characteristic length $N_*$, the accuracy cliff that separates reliable from unreliable generation. The hypothesis of independent per-step errors is rejected for every model and task we studied. The crossover is well described by a double-exponential accumulation law, $\mathrm{SAR}=\exp(-\beta_0 N\alpha^{N-1})$, whose crossover scale $N_*$ does not depend on the functional form chosen to fit it. To interpret this behaviour we introduce a minimal effective model in which step-correctness variables interact through dense random couplings and compete with an external field set by the prompt. Solved by direct enumeration, the model reproduces the super-linear error accumulation and the accuracy cliff qualitatively, and it assigns to each model--task pair two interpretable parameters, an intrinsic error rate and an error-accumulation factor.

Artificial Intelligence (cs.AI)

arXiv:2511.00763 [cs.AI]

(or arXiv:2511.00763v3 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2511.00763 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.