Tool Calling is Linearly Readable and Steerable in Language Models

Abstract:Language-model agents can take real actions by calling tools, so choosing the wrong tool can cause errors that are difficult to undo. Most evaluations only observe the tool choice after the model generates a call. We read this choice from the model before generation, and we steer it. For each tool, we average the model's hidden states from a few example requests to obtain a tool vector. Comparing a new request with these tool vectors predicts which tool it needs. The difference between two tool vectors gives a steering direction that can move the model toward a chosen tool without retraining. Across eight instruction-tuned models from 4B to 27B parameters, steering changes the generated call to a chosen target tool in 56-78% of held-out tool pairs, depending on the model. We also observe steering on real APIs from $\tau$-bench and ToolBench, though less reliably. At the final layer, steering could work simply by raising the score of the target tool name. To test this, we compare the steering direction with a direction that only raises that score, layer by layer. In the middle layers, on the three models we examine in depth, the steering direction switches more calls than the score-raising direction. So the tool vectors capture part of the tool choice before the final layer. The same tool vectors also help identify likely tool-selection errors before generation. Calls are more likely to be wrong when a request is similarly close to two tool vectors. Across four models, this signal achieves an AUROC of 0.61-0.78 and outperforms a first-token confidence baseline on three of them. Together, these results suggest that the model's hidden state provides a way to read, steer, and check tool choice before a call is made.

54 pages. v3: revised manuscript with new experiments

Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Software Engineering (cs.SE)

arXiv:2605.07990 [cs.CL]

(or arXiv:2605.07990v3 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2605.07990 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.