aiqbench analysis
This paper introduces DELTA, a training-free inference-time method for masked diffusion language models that achieves precise character-length control in text summarization without fine-tuning. By dynamically adjusting masked positions during the denoising process, the approach reduces length errors by an order of magnitude compared to existing baselines while maintaining high summary quality. This advancement addresses a specific limitation of autoregressive models and offers a practical solution for applications requiring strict output constraints like headline generation.
Training-Free Character-Length Control in Summarization with Diffusion Language Models
arXiv:2611.06371v1 Announce Type: new Abstract: Summarization with a fixed character length is important for tasks such as headline and article preview generation, where text must fit an exact space. Autoregressive language models (LMs) struggle with precise character-length control: training-based methods require task-specific fine-tuning, while zero-shot prompting strategies achieve only approximate compliance. We propose DELTA (Diffusion Expected Length Targeting Algorithm), a training-free inference-time method for masked diffusion LMs that estimates expected character length at each denoising step and dynamically inserts or removes masked positions to steer toward a target length. DELTA requires no fine-tuning and is designed for off-the-shelf masked diffusion LMs. Across three summarization benchmarks, it reduces character-length errors by an order of magnitude compared to baselines while maintaining competitive summary quality in both automatic and human evaluation.
Abstract:Summarization with a fixed character length is important for tasks such as headline and article preview generation, where text must fit an exact space. Autoregressive language models (LMs) struggle with precise character-length control: training-based methods require task-specific fine-tuning, while zero-shot prompting strategies achieve only approximate compliance. We propose DELTA (Diffusion Expected Length Targeting Algorithm), a training-free inference-time method for masked diffusion LMs that estimates expected character length at each denoising step and dynamically inserts or removes masked positions to steer toward a target length. DELTA requires no fine-tuning and is designed for off-the-shelf masked diffusion LMs. Across three summarization benchmarks, it reduces character-length errors by an order of magnitude compared to baselines while maintaining competitive summary quality in both automatic and human evaluation.
Computation and Language (cs.CL)
arXiv:2611.06371 [cs.CL]
(or arXiv:2611.06371v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2611.06371 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
Proceedings of EMNLP 2026, pp. 2803-2813
https://doi.org/10.18653/2026.emnlp-main.121 Focus to learn more DOI(s) linking to related resources
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 234 words