analiza aiqbench
This paper introduces LoopCD, a decoding technique designed to stabilize latent reasoning in Looped Language Models by contrasting logits across iterative refinement steps. By targeting unstable "hard" tokens at inference time without additional training, the method effectively mitigates loop instability and boosts performance on complex reasoning tasks. The approach offers a lightweight, efficient solution for enhancing the reliability of next-generation models that rely on recursive internal dynamics rather than explicit verbal chains.
LoopCD: Loop-wise Contrastive Decoding for Improving Reasoning in Looped Language Models
arXiv:2609.24196v1 Announce Type: new Abstract: Looped Language Models (LoopLMs) perform "latent reasoning" by recursively refining internal latent representations with shared weights, offering a more effective alternative to explicit verbal reasoning. Despite their effectiveness, we find that LoopLMs remain prone to loop instability: unstable refinement across iterations can produce localized uncertain "hard" tokens associated with reasoning errors. To address this, we propose LoopCD, loop-wise contrastive decoding that enhances the reasoning performance of LoopLMs by intervening on these tokens at inference time. Specifically, we exploit the internal dynamics of LoopLMs and contrast the logits from earlier iterations with logits from the last refined iteration to form the final sampling distribution. We find that this strategy is highly efficient, introducing only negligible inference overhead and requiring no additional training, while effectively improving reasoning performance by naturally refining reasoning-critical hard tokens. Extensive experiments show that our method improves the performance of recent representative LoopLMs across various reasoning tasks.
Abstract:Looped Language Models (LoopLMs) perform "latent reasoning" by recursively refining internal latent representations with shared weights, offering a more effective alternative to explicit verbal reasoning. Despite their effectiveness, we find that LoopLMs remain prone to loop instability: unstable refinement across iterations can produce localized uncertain "hard" tokens associated with reasoning errors. To address this, we propose LoopCD, loop-wise contrastive decoding that enhances the reasoning performance of LoopLMs by intervening on these tokens at inference time. Specifically, we exploit the internal dynamics of LoopLMs and contrast the logits from earlier iterations with logits from the last refined iteration to form the final sampling distribution. We find that this strategy is highly efficient, introducing only negligible inference overhead and requiring no additional training, while effectively improving reasoning performance by naturally refining reasoning-critical hard tokens. Extensive experiments show that our method improves the performance of recent representative LoopLMs across various reasoning tasks.
Computation and Language (cs.CL)
arXiv:2609.24196 [cs.CL]
(or arXiv:2609.24196v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.24196 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 244 cuvinte
Subiecte conexe
[AINews] not much happened today
Latent Space · 3 oct. 2026
How can LLM RL Work Despite Information-Theoretic Inefficiency
LessWrong · 16 oct. 2026
Sen. Adam Schiff on AI regulation, free speech, and impeaching Trump one more time
The Verge · 5 oct. 2026
[AINews] Opus 5.5 is good at explainer videos
Latent Space · 29 sept. 2026