analiza aiqbench
Latent Reward Steering introduces an adaptive inference-time framework that optimizes sparse autoencoder latent states to implicitly enhance cognitive behaviors in reasoning large language models. By training a reward model on final answer correctness, the method provides state-specific corrections only where fragile, thereby improving performance across various reasoning benchmarks without relying on predefined behavioral controls. This approach advances the field of machine learning by demonstrating how internal latent dynamics can be steered to fix reasoning errors more effectively than explicit behavior-level interventions.
Latent Reward Steering: An Adaptive Inference-Time Framework that Implicitly Promotes Cognitive Behaviors in Reasoning LLMs
arXiv:2606.00726v3 Announce Type: replace Abstract: Strong reasoning depends not only on model knowledge but also on how effectively cognitive behaviors are deployed during generation. Existing methods often rely on explicit behavior-level control, making them insufficiently adaptive when failures and required corrections vary across reasoning states, tasks, and models. To this end, we propose Latent Reward Steering (LRS), an adaptive inference-time framework that promotes cognitive behaviors by optimizing the sparse-autoencoder (SAE) latent states that implicitly carry them. Rather than relying on predefined cognitive behaviors or steering directions derived from them, LRS trains a latent reward model on reasoning traces by final answer correctness to estimate the quality of intermediate latent states. During inference, reward gradients provide state-specific correction directions for fragile latent states, while a reward and confidence gate restricts intervention to states the reward signal flags as fragile. Experiments on multiple reasoning LLM backbones and benchmarks show that \ours consistently improves performance over various baselines, and post-hoc analyses further indicate that \ours implicitly promotes good cognitive behaviors that fix the original reasoning errors. Code is available at: https://github.com/jiakanglee/Latent-Reward-Steering.
Abstract:Strong reasoning depends not only on model knowledge but also on how effectively cognitive behaviors are deployed during generation. Existing methods often rely on explicit behavior-level control, making them insufficiently adaptive when failures and required corrections vary across reasoning states, tasks, and models. To this end, we propose Latent Reward Steering (LRS), an adaptive inference-time framework that promotes cognitive behaviors by optimizing the sparse-autoencoder (SAE) latent states that implicitly carry them. Rather than relying on predefined cognitive behaviors or steering directions derived from them, LRS trains a latent reward model on reasoning traces by final answer correctness to estimate the quality of intermediate latent states. During inference, reward gradients provide state-specific correction directions for fragile latent states, while a reward and confidence gate restricts intervention to states the reward signal flags as fragile. Experiments on multiple reasoning LLM backbones and benchmarks show that \ours consistently improves performance over various baselines, and post-hoc analyses further indicate that \ours implicitly promotes good cognitive behaviors that fix the original reasoning errors. Code is available at: this https URL.
Artificial Intelligence (cs.AI)
arXiv:2606.00726 [cs.AI]
(or arXiv:2606.00726v3 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2606.00726 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 267 cuvinte
Subiecte conexe
ATLAS: Verifier-Guided Adaptive Latent Activation Steering for Efficient LLM Reasoning
arXiv · 9 iun. 2026
ATLAS: Verifier-Guided Adaptive Latent Activation Steering for Efficient LLM Reasoning
arXiv · 10 iun. 2026
Inference-time Policy Steering via Vision and Touch
arXiv · 16 iun. 2026
Developing Healthcare Robotics with GPU-Native Medical Physics Simulation
NVIDIA · 28 iul. 2026