aiqbench analysis
This paper introduces LEAF, a method that enhances Large Language Model reasoning in reinforcement learning with verifiable rewards by injecting dynamic memory and using self-distillation to improve exploration efficiency. By leveraging feedback from previous rollouts rather than relying solely on independent sampling, the approach achieves a fourfold acceleration in exploration while reducing computational costs compared to standard baselines. The technique demonstrates significant accuracy gains on unseen test sets, offering a promising direction for optimizing training dynamics in complex reasoning tasks.
Explore with Memory, Learn without It: In-Context Exploration and Self-Distillation for LLM Reasoning
arXiv:2610.17734v1 Announce Type: cross Abstract: Exploration bounds learning efficiency in reinforcement learning with verifiable rewards (RLVR). Current methods explore mostly through independent sampling, without fully using the information gathered along the way, such as known wrong answers and environment feedback. We propose LEAF, which injects a dynamic, problem-specific memory into the context before every rollout. The memory contains the most recent response, its reward, and other environment feedback. It is updated after every verifier call. We decouple policy updates from rollouts and update the policy through self-distillation. On four benchmarks and two open models, LEAF accelerates exploration by about 4x compared with a random sampling baseline. On unseen test sets, it improves accuracy over the strongest baseline by 1.59 and 1.83 percentage points on the two models, while using less compute than any RL baseline. These results highlight the potential of LEAF in improving both exploration and learning efficiency. Code is available at: https://github.com/on1262/LEAF
Abstract:Exploration bounds learning efficiency in reinforcement learning with verifiable rewards (RLVR). Current methods explore mostly through independent sampling, without fully using the information gathered along the way, such as known wrong answers and environment feedback. We propose LEAF, which injects a dynamic, problem-specific memory into the context before every rollout. The memory contains the most recent response, its reward, and other environment feedback. It is updated after every verifier call. We decouple policy updates from rollouts and update the policy through self-distillation. On four benchmarks and two open models, LEAF accelerates exploration by about 4x compared with a random sampling baseline. On unseen test sets, it improves accuracy over the strongest baseline by 1.59 and 1.83 percentage points on the two models, while using less compute than any RL baseline. These results highlight the potential of LEAF in improving both exploration and learning efficiency. Code is available at: this https URL
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2610.17734 [cs.LG]
(or arXiv:2610.17734v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.17734 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 248 words
Related coverage
What's Continual Learning, and Why Might We Expect To See It In Advanced LLM Agents?
LessWrong · Jun 12, 2026
How might continual learning affect safety and alignment?
LessWrong · Jun 13, 2026
[AINews] not much happened today
Latent Space · Sep 10, 2026
Continual learning might make your blocking monitors nearly useless
AI Alignment Forum · Sep 24, 2026