GRADE: Graph Representation of LLM Agent Dependency and Execution

Abstract:A trace records what an LLM agent did at each step. What is gained by also recording what each step relied on? GRADE represents a run as one typed graph: execution edges come free from the trace, and dependency edges are supplied, each graded observed, declared, or inferred. On six observed-dependency corpora spanning tool use, coding, and the web, we price the dependency layer against run size under a fixed logistic probe. Within corpus the layer adds failure-prediction signal on three corpora, though one increment disappears under task-grouped folds and another reverses under a cubic spline. In leave-one-corpus-out transfer the size-normalized dependency block stays above chance on every held-out corpus while run size inverts on two. That pattern belongs to the probe: under a cubic spline the same block falls below chance on four of the six. A preregistered control then holds the node set, the step order and the edge count fixed and varies only which earlier step each dependency edge attaches to. Within corpus it separates the recovered assignment from a degree-matched counterfeit on no corpus, with a six-corpus mean difference of -0.0003. In transfer, scored on the size-plus-dependency arm, only SWE-Gym clears Holm in the six-corpus attachment screen. SWE-Gym, tau-bench and web clear Holm in the separate six-corpus paired task bootstrap. The counterfeit scores higher on two corpora. Withholding each target's whole task family instead leaves no attachment-screen survivor among five targets and turns SWE-Gym's contrast negative at -0.036. Reuse the protocol: grade each edge, keep a run-size baseline, and score matched attachment controls before crediting structure.

30 pages, 7 figures, 13 tables. Code: this https URL

Machine Learning (cs.LG)

arXiv:2606.22741 [cs.LG]

(or arXiv:2606.22741v2 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2606.22741 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.