aiqbench analysis
DeepPlanner introduces a novel reinforcement learning framework that optimizes the planning stage of deep research agents by specifically targeting high-entropy decision points. By shaping token-level advantages with an entropy-based term, this method achieves state-of-the-art performance on complex long-horizon tasks while significantly reducing training budgets compared to vanilla RL approaches. This advancement addresses a critical gap in current LLM architectures where explicit planning mechanisms often remain under-optimized.
DeepPlanner: Scaling Planning Capability for Deep Research Agents via Advantage Shaping
arXiv:2510.12979v2 Announce Type: replace Abstract: Large language models (LLMs) augmented with multi-step reasoning and action generation abilities have shown promise in leveraging external tools to tackle complex tasks that require long-horizon planning. However, existing approaches either rely on implicit planning in the reasoning stage or introduce explicit planners without systematically addressing how to optimize the planning stage. As evidence, we observe that under vanilla reinforcement learning (RL), planning tokens exhibit significantly higher entropy than other action tokens, revealing uncertain decision points that remain under-optimized. To address this, we propose DeepPlanner, an end-to-end RL framework that effectively enhances the planning capabilities of deep research agents. Our approach shapes token-level advantage with an entropy-based term to allocate larger updates to high entropy tokens, and selectively upweights sample-level advantages for planning-intensive rollouts. Extensive experiments across seven deep research benchmarks demonstrate that DeepPlanner improves planning quality and achieves state-of-the-art results under a substantially lower training budget.
Abstract:Large language models (LLMs) augmented with multi-step reasoning and action generation abilities have shown promise in leveraging external tools to tackle complex tasks that require long-horizon planning. However, existing approaches either rely on implicit planning in the reasoning stage or introduce explicit planners without systematically addressing how to optimize the planning stage. As evidence, we observe that under vanilla reinforcement learning (RL), planning tokens exhibit significantly higher entropy than other action tokens, revealing uncertain decision points that remain under-optimized. To address this, we propose DeepPlanner, an end-to-end RL framework that effectively enhances the planning capabilities of deep research agents. Our approach shapes token-level advantage with an entropy-based term to allocate larger updates to high entropy tokens, and selectively upweights sample-level advantages for planning-intensive rollouts. Extensive experiments across seven deep research benchmarks demonstrate that DeepPlanner improves planning quality and achieves state-of-the-art results under a substantially lower training budget.
Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
arXiv:2510.12979 [cs.AI]
(or arXiv:2510.12979v2 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2510.12979 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 243 words
Related coverage
News
Quantum Computing Report · Jun 7, 2026
[AINews] not much happened today
Latent Space · Sep 10, 2026
[AINews] The Future of Latent Space
Latent Space · Sep 25, 2026
Latent reasoning architectures would likely undermine CoT, our strongest oversight tool
AI Alignment Forum · Sep 23, 2026