analiza aiqbench
This research introduces Entropy-Gradient Inversion as a geometric fingerprint to bridge the gap between observable token behavior and internal reasoning mechanisms in Large Reasoning Models. By integrating this signature into Correlation-Regularized Group Policy Optimization, the authors demonstrate improved reinforcement learning stability without relying on costly external verifiers. The findings offer a concrete methodological advance for optimizing model scalability and logical performance at the algorithmic level.
Entropy-Gradient Inversion: Moving Toward Internal Mechanism of Large Reasoning Models
arXiv:2605.17770v4 Announce Type: replace Abstract: The advancement of Large Reasoning Models (LRMs) has catalyzed a paradigm shift from reactive ``fast thinking'' text generation to systematic, step-by-step ``slow thinking'' reasoning, unlocking state-of-the-art performance in complex mathematical and logical tasks. However, the field faces \textit{the fundamental gap between token-level behavioral analysis and internal reasoning mechanisms, and the instability of reinforcement learning (RL) for reasoning optimization relying on costly external verifiers}. We identify and formally define \textbf{Entropy-Gradient Inversion}, a robust negative correlation between token entropy and logit gradients that acts as a definitive geometric fingerprint for LRM reasoning capability. Building on this, we propose \textbf{Correlation-Regularized Group Policy Optimization (CorR-PO)}, which embeds this inversion signature into RL reward regularization. Extensive experiments on various reasoning benchmarks across multiple model scales show CorR-PO consistently outperforms state-of-the-art baselines, confirming that stronger inversion directly correlates with superior reasoning performance.
Abstract:The advancement of Large Reasoning Models (LRMs) has catalyzed a paradigm shift from reactive ``fast thinking'' text generation to systematic, step-by-step ``slow thinking'' reasoning, unlocking state-of-the-art performance in complex mathematical and logical tasks. However, the field faces \textit{the fundamental gap between token-level behavioral analysis and internal reasoning mechanisms, and the instability of reinforcement learning (RL) for reasoning optimization relying on costly external verifiers}. We identify and formally define \textbf{Entropy-Gradient Inversion}, a robust negative correlation between token entropy and logit gradients that acts as a definitive geometric fingerprint for LRM reasoning capability. Building on this, we propose \textbf{Correlation-Regularized Group Policy Optimization (CorR-PO)}, which embeds this inversion signature into RL reward regularization. Extensive experiments on various reasoning benchmarks across multiple model scales show CorR-PO consistently outperforms state-of-the-art baselines, confirming that stronger inversion directly correlates with superior reasoning performance.
Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
arXiv:2605.17770 [cs.AI]
(or arXiv:2605.17770v4 [cs.AI] for this version)
Text integral sub licența deschisă a publicației · 151 cuvinte
Subiecte conexe
Thermodynamics and Objects
LessWrong · 23 sept. 2026
Entropy-Gradient Inversion: Moving Toward Internal Mechanism of Large Reasoning Models
arXiv · 12 iun. 2026
J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses
LessWrong · 8 oct. 2026
News
Quantum Computing Report · 7 iun. 2026