aiqbench analysis
This research investigates optimal routing strategies for multi-turn LLM tasks, demonstrating that delaying handoffs between models does not improve utility on standard benchmarks like BFCL or MultiWOZ and often increases costs. The study highlights that refining supervision targets from noisy success curves to monotone fits significantly boosts router performance and cost efficiency across both offline datasets and live deployments. By constructing capability-based training targets, the authors achieve substantial gains in trajectory pass scores while reducing operational expenses per instance.
Handoff Curves: Initial Selection, Later Handoff, and Supervision Targets in Multi-turn LLM Routing
arXiv:2610.15993v1 Announce Type: new Abstract: We study how initial model selection, later handoff, and training targets affect success and cost in LLM tasks that span multiple turns. We compare several router architectures using handoff curves, which record task success and cost as the handoff step changes. Our dataset contains 34,204 complete executions of three Qwen models across BFCL, MultiWOZ, and $\tau$-retail, providing offline supervision for routing decisions. In our primary comparison, with Qwen3.5 35B-A3B as the starting model, allowing a later handoff does not establish an improvement in utility over initial selection on BFCL or MultiWOZ, and increases cost on BFCL across a range of cost weights. Changing the supervision targets improves router performance on both our dataset and TwinRouterBench. On our dataset, replacing noisy training success curves with monotone fits raises success by 0.2 to 4.1 percentage points at the same cost across the reported operating points. On TwinRouterBench, we construct targets that describe the model capability needed for the remaining work and difficult steps ahead. Compared with training on the benchmark's per-step labels, these targets raise trajectory pass scores by 7.8 percentage points and cost savings by 6.3 percentage points on the full static bank. A live deployment of the router trained with these targets resolves 79 of 100 instances at a router cost of 0.59 US dollars per instance.
Abstract:We study how initial model selection, later handoff, and training targets affect success and cost in LLM tasks that span multiple turns. We compare several router architectures using handoff curves, which record task success and cost as the handoff step changes. Our dataset contains 34,204 complete executions of three Qwen models across BFCL, MultiWOZ, and $\tau$-retail, providing offline supervision for routing decisions. In our primary comparison, with Qwen3.5 35B-A3B as the starting model, allowing a later handoff does not establish an improvement in utility over initial selection on BFCL or MultiWOZ, and increases cost on BFCL across a range of cost weights. Changing the supervision targets improves router performance on both our dataset and TwinRouterBench. On our dataset, replacing noisy training success curves with monotone fits raises success by 0.2 to 4.1 percentage points at the same cost across the reported operating points. On TwinRouterBench, we construct targets that describe the model capability needed for the remaining work and difficult steps ahead. Compared with training on the benchmark's per-step labels, these targets raise trajectory pass scores by 7.8 percentage points and cost savings by 6.3 percentage points on the full static bank. A live deployment of the router trained with these targets resolves 79 of 100 instances at a router cost of 0.59 US dollars per instance.
Artificial Intelligence (cs.AI)
arXiv:2610.15993 [cs.AI]
(or arXiv:2610.15993v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2610.15993 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 311 words
Related coverage
J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses
LessWrong · Oct 8, 2026
Hegseth AutoWarCom: who builds it, what the money buys, and why its hardest problem after Minab is the record, not the model
livetradingnews.com · Oct 2, 2026
The Machines that Make the Machines
NVIDIA · Oct 7, 2026
Lattice-Based Cryptography Explained
Quantum Zeitgeist · Aug 15, 2026