aiqbench analysis
RADAR introduces a reinforcement learning approach to optimize speculative sampling in LLM inference, dynamically adjusting draft tree generation to reduce computational overhead. By treating token candidate selection as a Markov Decision Process, the method achieves significant speedups over standard auto-regressive decoding without sacrificing accuracy. This research advances the efficiency frontier for deploying large models in latency-sensitive applications.
RADAR: Accelerate Large Language Model Inference With RL-Based Dynamic Draft Trees
arXiv:2512.14069v2 Announce Type: replace Abstract: Inference with modern Large Language Models (LLMs) is expensive and slow, and speculative sampling has emerged as an effective solution to this problem. However, the number of calls to the draft model for generating candidate tokens in speculative sampling is a preset hyperparameter, lacking flexibility. To generate and utilize the candidate tokens more effectively, we propose RADAR, a novel speculative sampling method with RL-based dynamic draft trees. RADAR formulates the draft tree generation process as a Markov Decision Process (MDP) and employs offline reinforcement learning to train a prediction model, which enables real-time decisions on calls to the draft model, reducing redundant computations and further accelerating inference. Evaluations across three LLMs and four tasks show that RADAR achieves a speedup of 3.17x-4.82x over the auto-regressive decoding baseline. The code is available at https://github.com/minaduki-sora/RADAR.
Abstract:Inference with modern Large Language Models (LLMs) is expensive and slow, and speculative sampling has emerged as an effective solution to this problem. However, the number of calls to the draft model for generating candidate tokens in speculative sampling is a preset hyperparameter, lacking flexibility. To generate and utilize the candidate tokens more effectively, we propose RADAR, a novel speculative sampling method with RL-based dynamic draft trees. RADAR formulates the draft tree generation process as a Markov Decision Process (MDP) and employs offline reinforcement learning to train a prediction model, which enables real-time decisions on calls to the draft model, reducing redundant computations and further accelerating inference. Evaluations across three LLMs and four tasks show that RADAR achieves a speedup of 3.17x-4.82x over the auto-regressive decoding baseline. The code is available at this https URL.
Artificial Intelligence (cs.AI)
arXiv:2512.14069 [cs.AI]
(or arXiv:2512.14069v2 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2512.14069 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 227 words
Related coverage
NVIDIA GTC Taipei at COMPUTEX: Live Updates on What’s Next in AI
NVIDIA · May 21, 2026
[AINews] not much happened today
Latent Space · Sep 10, 2026
[AINews] not much happened today
Latent Space · Sep 18, 2026
From Build to Benchmark: ONNX Model Serving with Triton Inference Server on AMD GPUs
AMD · May 22, 2026