Divide-and-Conquer CoT: RL for Reducing Latency via Parallel Reasoning

Abstract:Long chain-of-thought reasoning (Long CoT) is now fundamental to state-of-the-art LLMs, especially in mathematical reasoning. However, LLM generation is highly sequential, and long CoTs lead to a high latency. We propose to train Divide-and-Conquer CoT (DC-CoT) to reduce the latency. With DC-CoT, the model can act as a director that identifies distinct subtasks that can be performed in parallel in its reasoning process, and then spawns workers to execute the subtasks. Our goal is to achieve high accuracy, with a low longest path length, which is a theoretical measure of the latency needed for the response. We start with a long CoT base model (DeepScaleR-1.5B-Preview), and first use SFT with a small curated demonstration set to initialize its ability to spawn workers in a certain format. Because SFT degrades the accuracy significantly, we design a multi-stage RL algorithm, with various data filtering strategies, to recover the accuracy while decreasing the longest path length. Across several benchmarks including AIME 2024 and HMMT 2025, DC-CoT achieves similar accuracy as DeepScaleR-1.5B-Preview while decreasing longest path length by 35-40%. Our code, SFT dataset and models are publicly available at this https URL.

59 pages, 16 figures. v2: Added appendices on wall-clock latency, RL runs with fewer stages, and qualitative analysis of rollouts. Corrected the Olympiad Bench result for DSR-12K/Maj-DSR-3 (v1 omitted 5 out of 674 problems during evaluation)

Machine Learning (cs.LG)

arXiv:2601.23027 [cs.LG]

(or arXiv:2601.23027v2 [cs.LG] for this version)