aiqbench analysis
InftyThink+ addresses the quadratic cost and context limitations of standard chain-of-thought reasoning by employing reinforcement learning to optimize iterative summarization and continuation strategies. This approach delivers a 21% accuracy boost on AIME24 benchmarks while simultaneously reducing inference latency and accelerating training convergence. The method represents a significant efficiency gain for deploying large reasoning models without sacrificing performance on out-of-distribution tasks.
InftyThink+: Effective and Efficient Infinite-Horizon Reasoning via Reinforcement Learning
arXiv:2602.06960v3 Announce Type: replace-cross Abstract: Large reasoning models achieve strong performance by scaling inference-time chain-of-thought, but this paradigm suffers from quadratic cost, context length limits, and degraded reasoning due to lost-in-the-middle effects. Iterative reasoning mitigates these issues by periodically summarizing intermediate thoughts, yet existing methods rely on supervised learning or fixed heuristics and fail to optimize when to summarize, what to preserve, and how to resume reasoning. We propose InftyThink+, an end-to-end reinforcement learning framework that optimizes the entire iterative reasoning trajectory, building on model-controlled iteration boundaries and explicit summarization. InftyThink+ adopts a two-stage training scheme with supervised cold-start followed by trajectory-level reinforcement learning, enabling the model to learn strategic summarization and continuation decisions. Experiments on DeepSeek-R1-Distill-Qwen-1.5B show that InftyThink+ improves accuracy by 21% on AIME24 and outperforms conventional long chain-of-thought reinforcement learning by a clear margin, while also generalizing better to out-of-distribution benchmarks. Moreover, InftyThink+ significantly reduces inference latency and accelerates reinforcement learning training, demonstrating improved reasoning efficiency alongside stronger performance.
Abstract:Large reasoning models achieve strong performance by scaling inference-time chain-of-thought, but this paradigm suffers from quadratic cost, context length limits, and degraded reasoning due to lost-in-the-middle effects. Iterative reasoning mitigates these issues by periodically summarizing intermediate thoughts, yet existing methods rely on supervised learning or fixed heuristics and fail to optimize when to summarize, what to preserve, and how to resume reasoning. We propose InftyThink+, an end-to-end reinforcement learning framework that optimizes the entire iterative reasoning trajectory, building on model-controlled iteration boundaries and explicit summarization. InftyThink+ adopts a two-stage training scheme with supervised cold-start followed by trajectory-level reinforcement learning, enabling the model to learn strategic summarization and continuation decisions. Experiments on DeepSeek-R1-Distill-Qwen-1.5B show that InftyThink+ improves accuracy by 21% on AIME24 and outperforms conventional long chain-of-thought reinforcement learning by a clear margin, while also generalizing better to out-of-distribution benchmarks. Moreover, InftyThink+ significantly reduces inference latency and accelerates reinforcement learning training, demonstrating improved reasoning efficiency alongside stronger performance.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2602.06960 [cs.CL]
(or arXiv:2602.06960v3 [cs.CL] for this version)
Full text under the publisher's open licence · 172 words
Related coverage
AXRP Episode 50 - Eli Lifland on AI 2027
LessWrong · Aug 3, 2026
The Alignment Problem: Computer Science Meets Analytic Philosophy
LessWrong · Oct 18, 2026
What's Continual Learning, and Why Might We Expect To See It In Advanced LLM Agents?
LessWrong · Jun 12, 2026
Humans Are Not Chinchillas: Revisiting “How Quick and Big Would A Software Intelligence Explosion Be?”
LessWrong · Oct 5, 2026