aiqbench analysis
This research addresses a critical scaling paradox in agentic systems, where expanding task pools typically degrades performance due to suboptimal evidence selection. The proposed Successive Scale Racing (SSR) framework resolves this by adaptively searching for optimal evidence combinations, enabling steady performance gains as the task pool grows while significantly reducing compute costs. It represents a meaningful advancement in optimizing agent instruction and tool usage through more efficient evolutionary search strategies.
Forging More Tasks into Better Harnesses: Agentic Harness Evolution via Successive Scale Racing
arXiv:2610.28815v1 Announce Type: new Abstract: Agentic harness evolution optimizes an agent's instructions, tools, and control flow by using its task execution experiences as evolving evidence. While more evolution tasks should intuitively improve performance, our preliminary study reveals a paradoxical failure mode: the performance of existing methods degrades as the task pool expands. Our further analysis demonstrates that this degradation stems from suboptimal evidence selection during optimization, as different evidence combinations lead to substantially different evolution outcomes. To address this issue, we propose Successive Scale Racing (SSR), a framework that adaptively searches for optimal evidence combinations in each evolution round. SSR constructs diverse evidence combinations based on the corresponding task quality and coverage, and efficiently races the resulting candidate harnesses under a small compute budget. Empirically, across three distinct benchmarks, SSR achieves state-of-the-art results in all six configurations. Notably, as the task pool grows from 40 to 200, SSR steadily improves the pass rate from 64.3% to 68.3%, while the baselines' performances all drop. Furthermore, its racing mechanism identifies near-optimal candidates while reducing validation executions by 60%.
Abstract:Agentic harness evolution optimizes an agent's instructions, tools, and control flow by using its task execution experiences as evolving evidence. While more evolution tasks should intuitively improve performance, our preliminary study reveals a paradoxical failure mode: the performance of existing methods degrades as the task pool expands. Our further analysis demonstrates that this degradation stems from suboptimal evidence selection during optimization, as different evidence combinations lead to substantially different evolution outcomes. To address this issue, we propose Successive Scale Racing (SSR), a framework that adaptively searches for optimal evidence combinations in each evolution round. SSR constructs diverse evidence combinations based on the corresponding task quality and coverage, and efficiently races the resulting candidate harnesses under a small compute budget. Empirically, across three distinct benchmarks, SSR achieves state-of-the-art results in all six configurations. Notably, as the task pool grows from 40 to 200, SSR steadily improves the pass rate from 64.3% to 68.3%, while the baselines' performances all drop. Furthermore, its racing mechanism identifies near-optimal candidates while reducing validation executions by 60%.
Artificial Intelligence (cs.AI)
arXiv:2610.28815 [cs.AI]
(or arXiv:2610.28815v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2610.28815 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 265 words
Related coverage
HarnessRisk: A Lifecycle-Oriented Benchmark for Agent Harness Safety
arXiv · Aug 19, 2026
Scaling Trajectories for Complex Tasks through Recursive Self-Rewrite
arXiv · Oct 6, 2026
A Blind Trust, the Bloody Thrust: When Attacker-Controlled Hook Updates Steer AI Agent Harnesses towards Malicious Behaviors
arXiv · Sep 5, 2026
Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agents
arXiv · Oct 15, 2026