analiza aiqbench
This research introduces $O^2$-CritiCuRL, a curriculum reinforcement learning framework designed to improve the reliability of multimodal large language models by distinguishing critical reasoning steps from redundant ones. By combining offline multi-rollout analysis with online progressive refinement, the method addresses spurious shortcut behaviors that undermine model interpretability without sacrificing efficiency.
Offline-Online Curriculum RL for Multimodal Reasoning
arXiv:2607.23700v1 Announce Type: new Abstract: Multimodal large language models exhibit capabilities on reasoning tasks, yet often produce flawed intermediate steps while yielding correct final answers. This behavior undermines interpretability and reliability, suggesting reliance on spurious shortcuts rather than faithful reasoning. Although efforts have explored step-level supervision, distinguishing decisive steps from redundant ones remains challenging. We propose $O^2$-CritiCuRL, a novel curriculum reinforcement learning framework that introduces critical-step awareness through an iterative offline-online paradigm. In the offline stage, $O^2$-CritiCuRL conducts multi-rollout analysis over step-annotated trajectories to estimate step-level importance, allowing the framework to distill critical reasoning steps and filter out redundant ones. In the online stage, we employ a progressive step-level reinforcement learning strategy, where truncated chains guide the model to infer missing steps and refine its reasoning, thereby sharpening its focus on critical steps and overcoming the limitations of static supervision. Extensive experiments on multimodal reasoning benchmarks show that our method achieves state-of-the-art performance while delivering superior training and inference efficiency. Code is available at https://github.com/kk0013/CritiCuRL.
Abstract:Multimodal large language models exhibit capabilities on reasoning tasks, yet often produce flawed intermediate steps while yielding correct final answers. This behavior undermines interpretability and reliability, suggesting reliance on spurious shortcuts rather than faithful reasoning. Although efforts have explored step-level supervision, distinguishing decisive steps from redundant ones remains challenging. We propose $O^2$-CritiCuRL, a novel curriculum reinforcement learning framework that introduces critical-step awareness through an iterative offline-online paradigm. In the offline stage, $O^2$-CritiCuRL conducts multi-rollout analysis over step-annotated trajectories to estimate step-level importance, allowing the framework to distill critical reasoning steps and filter out redundant ones. In the online stage, we employ a progressive step-level reinforcement learning strategy, where truncated chains guide the model to infer missing steps and refine its reasoning, thereby sharpening its focus on critical steps and overcoming the limitations of static supervision. Extensive experiments on multimodal reasoning benchmarks show that our method achieves state-of-the-art performance while delivering superior training and inference efficiency. Code is available at this https URL.
Artificial Intelligence (cs.AI)
arXiv:2607.23700 [cs.AI]
(or arXiv:2607.23700v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2607.23700 arXiv-issued DOI via DataCite (pending registration)
Text integral sub licența deschisă a publicației · 180 cuvinte
Subiecte conexe
Douyin Multimodal Embedding Model Technical Report
arXiv · 4 aug. 2026
MedPAIR: Measuring Whether Physicians and AI Agree on What Matters in Medical QA
Snorkel AI Blog · 2 oct. 2026
[AINews] not much happened today
Latent Space · 18 sept. 2026
Amazon is investing in the Lean Focused Research Organization
Amazon Science · 26 iul. 2026