analiza aiqbench
STRIDE addresses a critical bottleneck in Reinforcement Learning with Verifiable Rewards by introducing fine-grained credit assignment based on verifiable strategic patterns rather than sparse final-answer correctness. This approach distinguishes beneficial reasoning steps from harmful ones using discriminative estimation of n-gram preferences, leading to more precise optimization across diverse models and tasks including VLMs. The method effectively bridges the gap between intermediate signal noise and reliable reward shaping for advanced LLM training paradigms.
STRIDE: Strategic Trajectory Reasoning via Discriminative Estimation for Verifiable Reinforcement Learning
arXiv:2606.15866v1 Announce Type: cross Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has become an effective post-training paradigm for improving the reasoning abilities of large language models. However, existing RLVR methods typically rely on final-answer correctness to assign trajectory-level rewards, providing sparse supervision and treating all tokens uniformly regardless of their actual contribution to reasoning. Although recent studies introduce intermediate signals such as process rewards, high-entropy tokens, and semantic uncertainty, these signals are often not inherently verifiable and may fail to distinguish beneficial strategic patterns from harmful ones. To address this limitation, we propose STRIDE (Strategic Trajectory Reasoning with Discriminative Estimation), a fine-grained RLVR framework that derives strategic reasoning supervision from verifiable outcomes. STRIDE contrasts successful and failed trajectories within each response group to estimate the outcome-discriminative preference of each $n$-gram strategic pattern, and further combines this signal with reasoning saliency entropy to identify decision-relevant strategic patterns. These patterns are assigned differentiated advantage values during RL optimization, enabling more precise credit assignment while preserving the verifiability of RLVR. Extensive experiments demonstrate that STRIDE consistently improves reasoning performance across diverse models, tasks, and extended settings, including VLMs and agent-based systems.
Abstract:Reinforcement Learning with Verifiable Rewards (RLVR) has become an effective post-training paradigm for improving the reasoning abilities of large language models. However, existing RLVR methods typically rely on final-answer correctness to assign trajectory-level rewards, providing sparse supervision and treating all tokens uniformly regardless of their actual contribution to reasoning. Although recent studies introduce intermediate signals such as process rewards, high-entropy tokens, and semantic uncertainty, these signals are often not inherently verifiable and may fail to distinguish beneficial strategic patterns from harmful ones. To address this limitation, we propose STRIDE (Strategic Trajectory Reasoning with Discriminative Estimation), a fine-grained RLVR framework that derives strategic reasoning supervision from verifiable outcomes. STRIDE contrasts successful and failed trajectories within each response group to estimate the outcome-discriminative preference of each $n$-gram strategic pattern, and further combines this signal with reasoning saliency entropy to identify decision-relevant strategic patterns. These patterns are assigned differentiated advantage values during RL optimization, enabling more precise credit assignment while preserving the verifiability of RLVR. Extensive experiments demonstrate that STRIDE consistently improves reasoning performance across diverse models, tasks, and extended settings, including VLMs and agent-based systems.
Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2606.15866 [cs.AI]
(or arXiv:2606.15866v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2606.15866 arXiv-issued DOI via DataCite (pending registration)
Text integral sub licența deschisă a publicației · 204 cuvinte
Subiecte conexe
Memory as transformation: LETHE, a self-referential gan-inspired architecture
arXiv · 7 sept. 2026
RewardExplainer: Learning Reward Model Explanations from Counterfactual Preference Feedback
arXiv · 30 sept. 2026
Machine Learning Approaches to Decoding Topological Quantum Codes
arXiv · 19 aug. 2026
Adaptive Multi-Discriminator WGAN Framework for Resource-Constrained Internet of Vehicles Using Reinforcement Learning and Game Theory
arXiv · 9 oct. 2026