aiqbench analysis
This research introduces Variance-Calibrated Modulation (VCM), a training-free decoding technique that dynamically adjusts logit distributions to prevent LLMs from falling into repetitive generation loops known as the "likelihood trap." By leveraging real-time logit standard deviation and contextual probability measures, VCM enhances output diversity and reasoning accuracy without adding significant computational overhead. The method offers a practical improvement for existing inference pipelines, particularly when operating at higher decoding temperatures where coherence issues are most prevalent.
Breaking the Likelihood Trap: Variance-Calibrated Modulation for Large Language Model Decoding
arXiv:2606.22511v3 Announce Type: replace Abstract: In open-ended generation, LLMs frequently fall into the "likelihood trap", characterized by repetitive degeneration and vocabulary dullness, resulting in a discrepancy between machine-generated and human-written text. While post-hoc tail truncation (e.g., Top-p, Min-p) avoids sampling from the unreliable tail, it can misalign generation with human lexical preferences by over-sampling from the uncalibrated head; fixed scalar repetition penalties, in turn, ignore how the scale of the logit distribution varies across inference steps, which can disrupt semantic coherence. To address both shortcomings, we propose Variance-Calibrated Modulation (VCM), a training-free pre-decoding intervention. VCM directly reshapes the probability distribution prior to truncation via two dynamic mechanisms: (1) Contextual Searchlight via PMI, which naturally suppresses global stopwords and elevates context-evoked tokens, and (2) Adaptive Self-Debiasing, which utilizes real-time logit standard deviation to provide scale-invariant penalization. In experiments across open-ended generation, factual QA, and mathematical reasoning, we show that VCM consistently mitigates the likelihood trap. With negligible computational overhead, VCM integrates with existing decoding strategies, improving diversity and coherence and, particularly at higher decoding temperatures, reasoning accuracy. Our code is publicly available on GitHub: https://github.com/AetherDing/VCM
Abstract:In open-ended generation, LLMs frequently fall into the "likelihood trap", characterized by repetitive degeneration and vocabulary dullness, resulting in a discrepancy between machine-generated and human-written text. While post-hoc tail truncation (e.g., Top-p, Min-p) avoids sampling from the unreliable tail, it can misalign generation with human lexical preferences by over-sampling from the uncalibrated head; fixed scalar repetition penalties, in turn, ignore how the scale of the logit distribution varies across inference steps, which can disrupt semantic coherence. To address both shortcomings, we propose Variance-Calibrated Modulation (VCM), a training-free pre-decoding intervention. VCM directly reshapes the probability distribution prior to truncation via two dynamic mechanisms: (1) Contextual Searchlight via PMI, which naturally suppresses global stopwords and elevates context-evoked tokens, and (2) Adaptive Self-Debiasing, which utilizes real-time logit standard deviation to provide scale-invariant penalization. In experiments across open-ended generation, factual QA, and mathematical reasoning, we show that VCM consistently mitigates the likelihood trap. With negligible computational overhead, VCM integrates with existing decoding strategies, improving diversity and coherence and, particularly at higher decoding temperatures, reasoning accuracy. Our code is publicly available on GitHub: this https URL
Computation and Language (cs.CL); Machine Learning (stat.ML)
arXiv:2606.22511 [cs.CL]
(or arXiv:2606.22511v3 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.22511 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 277 words
Related coverage
News
Quantum Computing Report · Jul 25, 2026
The J-Space Debate, Agent Swarms, and Pacing Frontier AI - Digital Minds Newsletter #4
LessWrong · Sep 18, 2026
Jev: System One models for Prod, not God — with Diogo Almeida, CEO, TypeSafe AI
Latent Space · Sep 21, 2026
Compatibility-Aware Dynamic Fine-Tuning for Large Language Models
arXiv · Jun 12, 2026