aiqbench analysis
This article analyzes how LLM agent trajectories reveal hidden inefficiencies, such as excessive token usage and failed network routing attempts, that standard evaluation scores miss. It offers concrete strategies for optimizing product documentation and agent logic to reduce wasted compute resources during task execution.
Your best AX fixes are hiding in agent trajectories

All five agent runs produced a working script, and nothing in the eval score looked wrong. The trajectories told a different story. Every run read a DLL as raw bytes to find plugin names, and three scripts would quietly reroute a CI runner's network traffic. Here are the patterns to look for in your own agents' trajectories, and what to change in your product when you find them. The post Your best AX fixes are hiding in agent trajectories appeared first on Microsoft for Developers.
An eval score tells you whether the agent finished the task. To improve how agents work with your product, you need to see where they struggled, and that’s in the trajectory, the record of every step the agent took and why. Agents are persistent. They guess another URL after a 404 and try values until an error goes away, so the struggle rarely shows up in the score. We saw this when we evaluated Dev Proxy, a local proxy that intercepts your app’s HTTP requests. We asked GitHub Copilot Chat with Claude Sonnet 5.5 and no extensions, five times, to write a CI script that uses the Dev Proxy API to record which GitHub APIs a Node.js tool calls. All five scripts worked. Yet the trajectories showed 15 problems, and each run took 29 to 39 requests and up to 1.2 million tokens. Agents can trip over your technology in many ways. We picked the patterns we think you’re most likely to find in your own trajectories, from the obvious to the surprising, and for each one show what it looked like with Dev Proxy and which fix it pointed to. Agents often skip the search. They fetch the URL…
Excerpt shown under the publisher's RSS terms · 200 words · full text and figures are on the publisher's site
Related coverage
Hide-and-Seek in Trajectories: Discovering Failure Signals for VLA Runtime Monitoring
arXiv · Sep 28, 2026
SafeVLA-Bench: A Benchmark for the Success-Safety Gap in Vision-Language-Action Models
arXiv · Oct 1, 2026
I Prove, Therefore I Am: Spatiotemporal Multi-Party Computation
arXiv · Sep 23, 2026
How GUI Agents Fail: GraceBench for Execution Stability Diagnosis
arXiv · Oct 15, 2026