Implementing and Evaluating a Basic Per-Action Monitor for Safer Evals

Notes are unpolished research updates directly from METR researchers, and have gone through less review than our research posts. They do not necessarily reflect the views of METR as a whole. In light of recent incidents (e.g. those from OpenAI, Anthropic, and UK AISI), we developed and deployed a basic live per-action monitor to reduce the likelihood of incidents involving harmful actions from agents during our own evaluations. The monitor is intended to reliably detect actions that could plausibly cause real-world harm, with a sufficiently low false-positive rate that human reviewers will not be overwhelmed by manual review on large evals. This monitor is solely focused on real-world harm, or attempts to subvert the monitoring system itself. It is intended to ignore other nefarious things such as cheating, which we can scan for post-hoc. An LLM judge reviews each action from the agent before execution, and holds anything above a threshold for human review, halting the eval in the meantime. This post first sets out claims that we think would need to be made in order to make a strong argument for our monitoring system being effective. We give some notes on what evidence we have and what we’re missing,…