aiqbench analysis
Batch-Paced addresses the critical bottleneck of inference efficiency in discrete diffusion language models by optimizing compute allocation across batches without requiring retraining. This approach significantly improves overall batch quality and token confidence, offering a practical pathway to scale parallel generation methods for high-demand AI serving scenarios.
Batch-Paced: Fixed-Budget Batch Decoding in Discrete Diffusion Language Models
arXiv:2610.17917v1 Announce Type: new Abstract: As generative language models become more capable and widely used, inference demand grows, making efficient serving increasingly important. Discrete diffusion language models (dLLMs) offer a promising path to more efficient inference through parallel token generation. Prior dLLM decoding methods, however, focus on improving inference efficiency for individual sequences. We formulate fixed-budget dLLM batch decoding and introduce Batch-Paced, a training-free method that jointly optimizes the allocation of decoding compute across requests to improve overall batch quality. Across two dLLM backbones, five benchmarks and their representative mixtures, Batch-Paced improves batch performance under matched budgets; gains generally grow with batch size and largely persist across decoding configurations. Analysis of requests' decoding trajectories suggests that these gains arise largely by raising the committed-token confidence of lower-confidence requests. We further show that Batch-Paced's batch decode time remains competitive and that adding a decoding-depth allowance during Batch-Paced decoding can shorten it further at a modest performance cost.
Abstract:As generative language models become more capable and widely used, inference demand grows, making efficient serving increasingly important. Discrete diffusion language models (dLLMs) offer a promising path to more efficient inference through parallel token generation. Prior dLLM decoding methods, however, focus on improving inference efficiency for individual sequences. We formulate fixed-budget dLLM batch decoding and introduce Batch-Paced, a training-free method that jointly optimizes the allocation of decoding compute across requests to improve overall batch quality. Across two dLLM backbones, five benchmarks and their representative mixtures, Batch-Paced improves batch performance under matched budgets; gains generally grow with batch size and largely persist across decoding configurations. Analysis of requests' decoding trajectories suggests that these gains arise largely by raising the committed-token confidence of lower-confidence requests. We further show that Batch-Paced's batch decode time remains competitive and that adding a decoding-depth allowance during Batch-Paced decoding can shorten it further at a modest performance cost.
Computation and Language (cs.CL)
arXiv:2610.17917 [cs.CL]
(or arXiv:2610.17917v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2610.17917 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 246 words
Related coverage
[AINews] DeepSeek v4.1-Flash: 763B-P8B-D16B novel causal Encoder–Decoder architecture with vision marks the Return of the Whale
Latent Space · Sep 12, 2026
Benchmarking Kimi-K3 Across vLLM, SGLang, and ATOM on MI350X
AMD · Sep 22, 2026
Serving Masked Diffusion LLMs: Characterization and Design Principles from Real Hardware
arXiv · Aug 27, 2026
[AINews] Opus 5.5 is good at explainer videos
Latent Space · Sep 29, 2026