analiza aiqbench
SemBlock addresses a key limitation in diffusion language models by replacing fixed or delimiter-based block sizes with a dynamic framework driven by semantic boundaries. By training lightweight predictors on frozen hidden states using a new dataset derived from discourse units and reasoning steps, the method consistently outperforms existing decoding strategies on math and code benchmarks. This approach offers a practical path to improving the efficiency and coherence of iterative denoising models without requiring full retraining.
SemBlock: Semantic Boundary Dynamic Blocks for Diffusion LLMs
arXiv:2606.04964v1 Announce Type: new Abstract: Diffusion language models (DLMs) generate text through iterative denoising, and blockwise decoding improves their practicality by committing tokens in local blocks. However, existing blockwise methods typically rely on fixed block sizes or delimiter-based runtime signals, which do not necessarily align with semantic boundaries. In this paper, we propose SemBlock, a semantic-boundary-driven dynamic block decoding framework for diffusion LLMs. SemBlock formulates dynamic block construction as semantic boundary prediction and trains lightweight predictors on frozen LLaDA hidden states. To provide supervision, we construct SemBound, a semantic-boundary dataset that derives boundary labels from discourse units, reasoning steps, and implementation spans across natural language, math, and code tasks. During inference, SemBlock uses predicted boundary probabilities to select the ending position of each dynamic block. Experiments on GSM8K, IFEval, MATH, and HumanEval show that SemBlock consistently improves over fixed-block decoding and AdaBlock. Our code is publicly available: https://github.com/TH-AI-Lab-PKU/SemBlock.
Citește articolul complet pe arXiv