analiza aiqbench
Current coding-agent benchmarks fail to capture the "handoff debt" incurred when agents resume interrupted tasks, a critical gap between idealized single-agent tests and real-world collaborative workflows. The study demonstrates that providing structured context during handoffs can reduce agent events by up to 59% and prompt tokens by over 60%, proving that efficiency gains are consistent even if task completion rates vary. Consequently, future evaluation metrics must prioritize reporting the rediscovery cost of resuming work rather than solely focusing on final task resolution.
Handoff Debt: The Rediscovery Cost When Coding Agents Take Over Interrupted Tasks
arXiv:2606.02875v1 Announce Type: new Abstract: Coding-agent benchmarks evaluate whether a single uninterrupted agent can resolve a repository issue. Real software work is messier: tasks are interrupted, reassigned, reviewed, and resumed from partial states left by another agent or engineer. We study this missing dimension through \emph{handoff debt}: the rediscovery cost imposed when a predecessor's work is opaque or incomplete. Our takeover protocol interrupts a coding agent at deterministic handoff points, freezes the repository, and evaluates successor agents under four handoff views: repository state only, raw trace, summary notes, and structured notes. Across 75 source tasks, the protocol generates 181 handoff-point tasks and 724 takeover runs per successor model. Across three successor models, context-bearing handoffs reduce median agent events by 20--59\% and cumulative prompt tokens by 42--63\% relative to repository-only takeover. Solved-rate effects are smaller and model-dependent, but efficiency gains are consistent. These findings suggest that coding-agent evaluation should report not only whether a task is solved, but also how costly that work is for another agent to resume.
Abstract:Coding-agent benchmarks evaluate whether a single uninterrupted agent can resolve a repository issue. Real software work is messier: tasks are interrupted, reassigned, reviewed, and resumed from partial states left by another agent or engineer. We study this missing dimension through \emph{handoff debt}: the rediscovery cost imposed when a predecessor's work is opaque or incomplete. Our takeover protocol interrupts a coding agent at deterministic handoff points, freezes the repository, and evaluates successor agents under four handoff views: repository state only, raw trace, summary notes, and structured notes. Across 75 source tasks, the protocol generates 181 handoff-point tasks and 724 takeover runs per successor model. Across three successor models, context-bearing handoffs reduce median agent events by 20--59\% and cumulative prompt tokens by 42--63\% relative to repository-only takeover. Solved-rate effects are smaller and model-dependent, but efficiency gains are consistent. These findings suggest that coding-agent evaluation should report not only whether a task is solved, but also how costly that work is for another agent to resume.
Artificial Intelligence (cs.AI)
arXiv:2606.02875 [cs.AI]
(or arXiv:2606.02875v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2606.02875 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 257 cuvinte
Subiecte conexe
Handoff Debt: The Rediscovery Cost When Coding Agents Take Over Interrupted Tasks
arXiv · 1 sept. 2026
Why Software Automation Is Hard
LessWrong · 6 iun. 2026
[AINews] High Taste Testers Ep. 1
Latent Space · 17 oct. 2026

Defaulting on tech debt: When the bill comes due, AI is the collector
Sysdig Blog · 12 aug. 2026