aiqbench analysis
RoutePack introduces a novel optimization framework that jointly coordinates expert placement and rollout-trajectory scheduling to resolve coupled load-balancing bottlenecks in Mixture-of-Experts reinforcement learning. By leveraging replay-guided planning, the method achieves significant throughput gains on large-scale hybrid-attention models without compromising training-state consistency. This work directly addresses critical infrastructure challenges for scaling MoE architectures in high-performance distributed training environments.
RoutePack: Replay-Guided Expert Placement and Rollout Trajectory Scheduling for MoE Reinforcement Learning
arXiv:2608.12146v2 Announce Type: replace-cross Abstract: Reinforcement learning (RL) post-training of Mixture-of-Experts (MoE) models couples two load-balancing problems. Rollout-trajectory composition determines attention and dense feed-forward work across data-parallel (DP) replicas, while token routing determines expert work across expert-parallel ranks. Optimizing either side independently can worsen the other. Routing replay exposes trajectory lengths and layer-wise expert demand before training, creating an opportunity to coordinate both controls. We present RoutePack, a replay-guided planner that coordinates expert placement and rollout-trajectory scheduling. Its key insight is that placement determines a batchwise expert-rank load bound that trajectory scheduling alone cannot lower, while scheduling controls the hotspots within individual execution micro-steps. RoutePack incrementally adjusts expert placement while preserving expert residency, then uses parallel population annealing to reschedule complete trajectories into token-capped microbatches. Its objective jointly balances dense-side work and layer-wise expert peaks under a fixed, capacity-feasible micro-step count and the deployed synchronization groups. The planner preserves logical token routing and keeps placement fixed throughout forward and backward execution to maintain training-state consistency. We evaluate mixed-task RL post-training of two hybrid-attention MoE models (7.9B and 124B parameters) across six NVIDIA H20-3e configurations spanning 8K-32K contexts. Within the Megatron training region, RoutePack achieves 4.44-11.85% higher throughput than a DP-local first-fit-decreasing baseline and 1.33-5.28% higher throughput than expert rerouting alone. Additional evaluations of the 7.9B model on H200 and H20 retain positive gains. We separately characterize planning and admission costs.
Abstract:Reinforcement learning (RL) post-training of Mixture-of-Experts (MoE) models couples two load-balancing problems. Rollout-trajectory composition determines attention and dense feed-forward work across data-parallel (DP) replicas, while token routing determines expert work across expert-parallel ranks. Optimizing either side independently can worsen the other. Routing replay exposes trajectory lengths and layer-wise expert demand before training, creating an opportunity to coordinate both controls. We present RoutePack, a replay-guided planner that coordinates expert placement and rollout-trajectory scheduling. Its key insight is that placement determines a batchwise expert-rank load bound that trajectory scheduling alone cannot lower, while scheduling controls the hotspots within individual execution micro-steps. RoutePack incrementally adjusts expert placement while preserving expert residency, then uses parallel population annealing to reschedule complete trajectories into token-capped microbatches. Its objective jointly balances dense-side work and layer-wise expert peaks under a fixed, capacity-feasible micro-step count and the deployed synchronization groups. The planner preserves logical token routing and keeps placement fixed throughout forward and backward execution to maintain training-state consistency. We evaluate mixed-task RL post-training of two hybrid-attention MoE models (7.9B and 124B parameters) across six NVIDIA H20-3e configurations spanning 8K-32K contexts. Within the Megatron training region, RoutePack achieves 4.44-11.85% higher throughput than a DP-local first-fit-decreasing baseline and 1.33-5.28% higher throughput than expert rerouting alone. Additional evaluations of the 7.9B model on H200 and H20 retain positive gains. We separately characterize planning and admission costs.
Distributed, Parallel, and Cluster Computing (cs.DC); Machine Learning (cs.LG)
arXiv:2608.12146 [cs.DC]
(or arXiv:2608.12146v2 [cs.DC] for this version)
https://doi.org/10.48550/arXiv.2608.12146 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 323 words
Related coverage
[AINews] not much happened today
Latent Space · Oct 3, 2026
ATOM: Unlocking Extreme AMD Instinct Inference with Software-Hardware Co-Optimization
AMD · Jun 15, 2026
Granite 4.2 LLMs: How They're Built
Hugging Face · Aug 25, 2026
How can LLM RL Work Despite Information-Theoretic Inefficiency
LessWrong · Oct 16, 2026