analiza aiqbench
The paper proposes a data-selection method for reinforcement learning with verifiable rewards (RLVR) in math reasoning, arguing that some failed problems are recoverable through the model's own trajectory search and are worth targeting for training. The reported gains are modest: a 1.45-point average improvement on MATH500 and WE-MATH for one 8B model, and a 0.85-point edge over random selection in a single-problem setting. Readers should treat these as preliminary results from a single model family until they are replicated.
ReSI-Math: Exploring Recoverable Capability Frontiers for Efficient Reasoning Improvement
arXiv:2611.04787v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has become an effective approach for improving mathematical reasoning in large language models. However, a fundamental question remains: when a model fails to solve a problem, does it truly lack the underlying capability? In this work, we reveal that direct failure does not necessarily imply inability. Some reasoning failures can be recovered by exploring the model's own trajectories, exposing a recoverable capability frontier: a policy-dependent region where solutions are not reliably generated through direct decoding but can be elicited through trajectory-level search. Based on this observation, we introduce ReSI-Math, a framework that explores recoverable capability frontiers and converts recovered capabilities into data-efficient RLVR supervision. ReSI-Math uses Self-Judge-guided evolutionary search to identify recoverable failures under a fixed policy and adaptively selects frontier problems for sequential RLVR as the policy evolves. On Qwen3-VL-8B, correct-incorrect trajectory crossover produces 272 verifier-confirmed correct offspring from 621 attempts (43.8%). After three sequential one-shot RLVR stages, the MATH500 and WE-MATH average improves from 78.71 to 80.16. Under matched single-problem RLVR, random selection reaches only 53.31 versus 54.16 for frontier-guided Q1.
Abstract:Reinforcement learning with verifiable rewards (RLVR) has become an effective approach for improving mathematical reasoning in large language models. However, a fundamental question remains: when a model fails to solve a problem, does it truly lack the underlying capability? In this work, we reveal that direct failure does not necessarily imply inability. Some reasoning failures can be recovered by exploring the model's own trajectories, exposing a recoverable capability frontier: a policy-dependent region where solutions are not reliably generated through direct decoding but can be elicited through trajectory-level search. Based on this observation, we introduce ReSI-Math, a framework that explores recoverable capability frontiers and converts recovered capabilities into data-efficient RLVR supervision. ReSI-Math uses Self-Judge-guided evolutionary search to identify recoverable failures under a fixed policy and adaptively selects frontier problems for sequential RLVR as the policy evolves. On Qwen3-VL-8B, correct-incorrect trajectory crossover produces 272 verifier-confirmed correct offspring from 621 attempts (43.8%). After three sequential one-shot RLVR stages, the MATH500 and WE-MATH average improves from 78.71 to 80.16. Under matched single-problem RLVR, random selection reaches only 53.31 versus 54.16 for frontier-guided Q1.
20 pages, 7 figures, 14 tables. Project page: this https URL
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2611.04787 [cs.CL]
(or arXiv:2611.04787v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2611.04787 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 288 cuvinte
Subiecte conexe
Operational Abstractions of Neural Network Concepts via Topological Representations
arXiv · 7 oct. 2026
Faithful Activation Verbalization: Reducing Hallucinations in LLM Representation Interpretation
arXiv · 30 sept. 2026
Recoverability and Verifier Strength: Two Measured Preconditions for Unattended Agent Self-Improvement
arXiv · 21 oct. 2026
Native Geometry vs. Linear Recoverability: How Alignment Changes Platonic Scaling
arXiv · 19 oct. 2026