aiqbench analysis
This paper introduces SWITCH, a framework that enables the optimization and mechanistic analysis of latent chain-of-thought reasoning using standard on-policy reinforcement learning. By utilizing discrete boundary tokens to manage transitions between visible and latent states, the authors demonstrate that hidden-state recurrence can be effectively trained while preserving causal interpretability of internal computations. The work advances our understanding of how RL shapes model internals without requiring specialized or opaque training procedures.
Demystifying Hidden-State Recurrence: Switchable Latent Reasoning with On-Policy Reinforcement Learning
arXiv:2606.13106v2 Announce Type: replace-cross Abstract: Latent chain-of-thought compresses reasoning through continuous hidden-state recurrence, but existing approaches are difficult to optimize with standard on-policy reinforcement learning (RL) and hard to analyze mechanistically. We introduce SWITCH, a switchable latent reasoning framework that addresses both challenges with two discrete boundary tokens: enters latent reasoning and exits it. These discrete anchors make policy ratios well-defined for standard on-policy RL while providing natural intervention points for mechanistic analysis. We train SWITCH with a visible-to-latent curriculum and Switch-GRPO, which propagates gradients through recurrent latent computation. SWITCH consistently outperforms prior hidden-state-recurrence approaches at similar model scales. Mechanistic analysis reveals that (i) implements a localized, learned switching policy rather than a stylistic artifact; (ii) the latent step performs problem-specific, causally important computation; and (iii) this computation is concentrated in a single hidden-state transition upon entering latent mode. Together, these results show that hidden-state-recurrence latent reasoning can be both effectively optimized with on-policy RL and directly analyzed mechanistically, enabling us to study how RL reshapes internal reasoning computation. Code is available at https://github.com/LARK-AI-Lab/SWITCH.
Abstract:Latent chain-of-thought compresses reasoning through continuous hidden-state recurrence, but existing approaches are difficult to optimize with standard on-policy reinforcement learning (RL) and hard to analyze mechanistically. We introduce SWITCH, a switchable latent reasoning framework that addresses both challenges with two discrete boundary tokens: <swi> enters latent reasoning and </swi> exits it. These discrete anchors make policy ratios well-defined for standard on-policy RL while providing natural intervention points for mechanistic analysis. We train SWITCH with a visible-to-latent curriculum and Switch-GRPO, which propagates gradients through recurrent latent computation. SWITCH consistently outperforms prior hidden-state-recurrence approaches at similar model scales. Mechanistic analysis reveals that (i) <swi> implements a localized, learned switching policy rather than a stylistic artifact; (ii) the latent step performs problem-specific, causally important computation; and (iii) this computation is concentrated in a single hidden-state transition upon entering latent mode. Together, these results show that hidden-state-recurrence latent reasoning can be both effectively optimized with on-policy RL and directly analyzed mechanistically, enabling us to study how RL reshapes internal reasoning computation. Code is available at this https URL.
Machine Learning (cs.LG); Computation and Language (cs.CL)
arXiv:2606.13106 [cs.LG]
(or arXiv:2606.13106v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.13106 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 270 words
Related coverage
Demystifying Hidden-State Recurrence: Switchable Latent Reasoning with On-Policy Reinforcement Learning
arXiv · Jun 12, 2026
Latent reasoning architectures would likely undermine CoT, our strongest oversight tool
AI Alignment Forum · Sep 23, 2026
Reason Through the Latent! Making Latent Visual Reasoning Necessary
arXiv · Sep 9, 2026
J-CoT: Chain-of-Thought in J-Space
arXiv · Jul 28, 2026