NVIDIA NeMo Relay traces how AI agents complete tasks

NVIDIA NeMo Relay now provides a way to inspect the detailed steps an AI agent takes to complete a task, revealing inefficiencies hidden behind successful outcomes. Even a correct final answer can mask problems; for example. Developers can now use NeMo Relay, integrated natively with the Hermes Agent runtime, to examine event streams and trajectories, including model and tool calls, errors and token use, to understand how an agent achieves a result. The technology is demonstrated in a Hermes ToolPerf case study, evaluating harness changes across multiple runs using OpenTelemetry traces explored in Arize Phoenix. The potential for subtle inefficiencies in AI agents is now under scrutiny with the introduction of detailed execution tracing, as demonstrated by the Agent Trajectory Interchange Format (ATIF), a step-by-step JSON record of agent interactions and observations. This format assembles data from lifecycle events, allowing developers to move beyond simply verifying a final answer and instead analyze the process by which an agent arrives at that conclusion. A specific example of this hidden inefficiency is when an agent repeatedly fetches the same content,…