aiqbench analysis
This arXiv preprint proposes an inference-time method that repairs tool-use plans from small language models (1.5B to 4B parameters) by executing them against live tool servers and using the errors to guide edits. The reported gains, from about 3% to 17-24% execution success on MCP-Bench, are substantial, but they come from one benchmark, four planners, and held-out tasks, so generalization to other tool ecosystems is untested. For readers, the more transferable finding is that execution feedback can beat fine-tuning on the same small dataset (177 demonstrations, 624 preference pairs), at a fraction of ReAct's token cost.
EvoFlux: Inference-Time Evolution of Executable Tool Workflows for Compact Agents
arXiv:2606.12674v2 Announce Type: replace Abstract: Small language models (1.5B-4B parameters) offer low-cost, low-latency tool use but often produce plausible plans that fail during execution due to unavailable tools, missing arguments, or broken dependencies. We introduce EvoFlux, an inference-time search method that repairs tool plans through an evolutionary loop over structured graphs of tool calls and dependencies. EvoFlux executes candidate plans against live tool servers and uses execution errors to guide targeted edits, including replacing tools, correcting parameters, inserting missing steps, and removing redundant ones. On held-out MCP-Bench tasks spanning 28 live servers and 250 tools, EvoFlux improves execution success from roughly 3% to 17-24% across four small planners. SFT and SFT+DPO trained on the same 177 demonstrations and 624 preference pairs match or underperform the zero-shot baseline, while ReAct reaches higher peaks on stronger planners at 4-20x greater planner-token cost per solved task. Execution-guided inference-time repair reliably improves small tool-using agents when supervision is limited.
Abstract:Small language models (1.5B-4B parameters) offer low-cost, low-latency tool use but often produce plausible plans that fail during execution due to unavailable tools, missing arguments, or broken dependencies. We introduce EvoFlux, an inference-time search method that repairs tool plans through an evolutionary loop over structured graphs of tool calls and dependencies. EvoFlux executes candidate plans against live tool servers and uses execution errors to guide targeted edits, including replacing tools, correcting parameters, inserting missing steps, and removing redundant ones. On held-out MCP-Bench tasks spanning 28 live servers and 250 tools, EvoFlux improves execution success from roughly 3% to 17-24% across four small planners. SFT and SFT+DPO trained on the same 177 demonstrations and 624 preference pairs match or underperform the zero-shot baseline, while ReAct reaches higher peaks on stronger planners at 4-20x greater planner-token cost per solved task. Execution-guided inference-time repair reliably improves small tool-using agents when supervision is limited.
Artificial Intelligence (cs.AI)
arXiv:2606.12674 [cs.AI]
(or arXiv:2606.12674v2 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2606.12674 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 242 words
Related coverage
Evoflux: Inference-Time Evolution of Executable Tool Workflows for Compact Agents
arXiv · Jun 12, 2026
[AINews] Meta Muse Wonder & Wonder Full: MSL’s Claude 5.5-tier Frontier Model
Latent Space · Oct 13, 2026
NVIDIA GTC Taipei at COMPUTEX: Live Updates on What’s Next in AI
NVIDIA · May 21, 2026
[AINews] DeepSeek v4.1-Flash: 763B-P8B-D16B novel causal Encoder–Decoder architecture with vision marks the Return of the Whale
Latent Space · Sep 12, 2026