aiqbench analysis
This arXiv preprint proposes using expert routing statistics in Mixture-of-Experts models as a process-confidence signal for multi-step reasoning, and uses that signal both to trigger regeneration and to auto-label training data for a smaller process reward model. The reported gains are the authors' own and unreviewed, so the key questions for readers are how the routing signal compares with stronger baselines and whether it holds beyond the tested MoE architectures. If it does, it could reduce reliance on costly human or rollout-based step labels.
A Strong MoE Is All You Need for Process Supervision
arXiv:2610.29498v1 Announce Type: cross Abstract: Process reward models (PRMs) provide fine-grained supervision for multi-step reasoning, yet obtaining reliable process labels remains a central bottleneck. Existing approaches typically rely on human annotation, rollout-based estimation, or auxiliary verification. In this work, we explore a different source of process supervision natively exposed by Mixture-of-Experts (MoE) models: expert routing. We show that simple routing statistics provide an intrinsic process-confidence signal that is competitive with conventional output-logit confidence. Building on this observation, we construct routing confidence by combining overall routing mass, head routing mass, and within-step routing variation, and introduce RoCo-Rethink, which identifies the least confident reasoning step and selectively regenerates the subsequent trajectory. We further introduce RoCo-PRM, which uses a strong MoE to automatically annotate reasoning trajectories with routing-derived step scores and distills them into a standalone PRM. Across multiple MoE architectures and diverse reasoning tasks, both RoCo-Rethink and RoCo-PRM consistently outperform strong baselines, while RoCo-PRM transfers effectively across both model architectures and tasks. Our findings suggest that expert routing is more than a computational dispatch mechanism: it provides an architecture-native source of process supervision for reasoning.
Abstract:Process reward models (PRMs) provide fine-grained supervision for multi-step reasoning, yet obtaining reliable process labels remains a central bottleneck. Existing approaches typically rely on human annotation, rollout-based estimation, or auxiliary verification. In this work, we explore a different source of process supervision natively exposed by Mixture-of-Experts (MoE) models: expert routing. We show that simple routing statistics provide an intrinsic process-confidence signal that is competitive with conventional output-logit confidence. Building on this observation, we construct routing confidence by combining overall routing mass, head routing mass, and within-step routing variation, and introduce RoCo-Rethink, which identifies the least confident reasoning step and selectively regenerates the subsequent trajectory. We further introduce RoCo-PRM, which uses a strong MoE to automatically annotate reasoning trajectories with routing-derived step scores and distills them into a standalone PRM. Across multiple MoE architectures and diverse reasoning tasks, both RoCo-Rethink and RoCo-PRM consistently outperform strong baselines, while RoCo-PRM transfers effectively across both model architectures and tasks. Our findings suggest that expert routing is more than a computational dispatch mechanism: it provides an architecture-native source of process supervision for reasoning.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2610.29498 [cs.CL]
(or arXiv:2610.29498v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2610.29498 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 276 words
Related coverage
Reward-SQL: Boosting Text-to-SQL via Stepwise Execution-Aware Reasoning and Process-Supervised Rewards
arXiv · Jun 15, 2026
Combining Hierarchical Cognitive Process with Process Supervision for Interpretable Scene Safety Understanding
arXiv · Sep 23, 2026
Latent Decision Flow Rewards for Clinical Guideline Reasoning
arXiv · Oct 31, 2026
TagPR: Tag-Guided Process Supervision for Personalization Reasoning in Large Language Models
arXiv · Oct 1, 2026