aiqbench analysis
This paper introduces LEAF, a retrospective reinforcement learning method that improves credit assignment in speech-aware large language models by leveraging shared prefixes without requiring online branching. By assigning span-level advantages based on descendant rewards, the approach allows smaller models to consistently outperform full-parameter baselines in tasks like speech question answering and translation. The work addresses a key inefficiency in current GRPO-style training methods, offering a more structured way to handle complex decision boundaries in generative tasks.
LEAF: Growing Trees Without Branching for Speech-Aware Large Language Model Post-Training
arXiv:2606.07610v2 Announce Type: replace-cross Abstract: State-of-the-art GRPO-style methods for speech-aware large language model post-training suffer from coarse credit assignment, broadcasting the same terminal-reward advantage to every token in a response. This ignores useful structure within rollout batches, where speech-conditioned completions often share prefixes before diverging at important decisions. We propose Low-rank Exploration with Adaptive Forking (LEAF), a retrospective tree-based RL method that recovers this structure without online branching or additional decoding. LEAF samples complete responses, selects high-surprisal boundaries, groups responses by shared prefixes, and assigns span-level advantages using descendant rewards. We theoretically justify LEAF's span-level credit assignment and boundary-selection design. Empirically, LEAF improves over GRPO across speech question answering and speech translation benchmarks under the same rollout and low-rank adaptation budget. Notably, smaller LEAF-trained models consistently outperform current state-of-art, full-parameter baselines.
Abstract:State-of-the-art GRPO-style methods for speech-aware large language model post-training suffer from coarse credit assignment, broadcasting the same terminal-reward advantage to every token in a response. This ignores useful structure within rollout batches, where speech-conditioned completions often share prefixes before diverging at important decisions. We propose Low-rank Exploration with Adaptive Forking (LEAF), a retrospective tree-based RL method that recovers this structure without online branching or additional decoding. LEAF samples complete responses, selects high-surprisal boundaries, groups responses by shared prefixes, and assigns span-level advantages using descendant rewards. We theoretically justify LEAF's span-level credit assignment and boundary-selection design. Empirically, LEAF improves over GRPO across speech question answering and speech translation benchmarks under the same rollout and low-rank adaptation budget. Notably, smaller LEAF-trained models consistently outperform current state-of-art, full-parameter baselines.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
arXiv:2606.07610 [cs.LG]
(or arXiv:2606.07610v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.07610 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 225 words
Related coverage
LEAF: Growing Trees Without Branching for Speech-Aware Large Language Model Post-Training
arXiv · Jun 9, 2026
PagedAttention vs. RadixAttention: Optimizing LLM KV Cache Management
Analytics Vidhya · Aug 21, 2026
SPIR-V on ROCm: A Portable IR for AMD GPUs
AMD · Jul 20, 2026
Notes on Algorithms
LessWrong · Jun 10, 2026