Length Value Model: Scalable Value Pretraining for Token-Level Length Modeling

arXiv:2604.27039v2 Announce Type: replace Abstract: Tokens are the fundamental units of computation in modern autoregressive models, and generation length directly influences both inference cost and reasoning performance. Despite its importance, existing approaches model length primarily at the coarse sequence level. We introduce the Length Value Model (LenVM), a token-level framework that estimates the remaining generation length at every decoding step. By formulating length modeling as a value estimation problem and assigning a constant negative reward to each generated token, LenVM predicts a bounded, discounted return that is a monotone proxy for the remaining generation horizon. This value formulation provides annotation-free, dense, unbiased, and scalable supervision. Experiments on LLMs and VLMs show that LenVM supports exact control, continuous performance--efficiency steering, length prediction, and interpretation. On LIFEBench-token, it raises the exact-length score of Qwen2.5-7B-Instruct from $30.9$ to $64.8$ in one pass and reaches $83.6$ when combined with LCG. Near $200$ tokens on GSM8K, LenVM retains about $63\%$ Pass@1 versus $6\%$ under a hard token budget. Across other settings, LenVM improves matched-length quality over prompting and EOS calibration, while its length-prediction accuracy improves with scale and exceeds specialized predictors. Its token-level values also identify markers of shifts between longer and shorter trajectories. These results establish generation length as an effective scalable token-level value signal for control, prediction, and future value-based training.

Abstract:Tokens are the fundamental units of computation in modern autoregressive models, and generation length directly influences both inference cost and reasoning performance. Despite its importance, existing approaches model length primarily at the coarse sequence level. We introduce the Length Value Model (LenVM), a token-level framework that estimates the remaining generation length at every decoding step. By formulating length modeling as a value estimation problem and assigning a constant negative reward to each generated token, LenVM predicts a bounded, discounted return that is a monotone proxy for the remaining generation horizon. This value formulation provides annotation-free, dense, unbiased, and scalable supervision. Experiments on LLMs and VLMs show that LenVM supports exact control, continuous performance--efficiency steering, length prediction, and interpretation. On LIFEBench-token, it raises the exact-length score of Qwen2.5-7B-Instruct from $30.9$ to $64.8$ in one pass and reaches $83.6$ when combined with LCG. Near $200$ tokens on GSM8K, LenVM retains about $63\%$ Pass@1 versus $6\%$ under a hard token budget. Across other settings, LenVM improves matched-length quality over prompting and EOS calibration, while its length-prediction accuracy improves with scale and exceeds specialized predictors. Its token-level values also identify markers of shifts between longer and shorter trajectories. These results establish generation length as an effective scalable token-level value signal for control, prediction, and future value-based training.

Computation and Language (cs.CL)

arXiv:2604.27039 [cs.CL]

(or arXiv:2604.27039v2 [cs.CL] for this version)

aiqbench analysis

Length Value Model: Scalable Value Pretraining for Token-Level Length Modeling

Related coverage