Is METR A Meaningful Check On Anthropic?

Each frontier AI company commits to giving ongoing, employee-like access to a team of embedded third-party evaluators (such as METR), whose role is to verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes. This is the key step for verifiability of any pacing commitments, and has precedent in the banking industry, which sometimes involves regulatory “supervisors” embedded along with employees. Anthropic is unilaterally committing to this step now. We intend this to be part of a broader push to redouble efforts on our safety and alignment work. — Dario Amodei, “We Must Pace the Frontier”, September 2026. METR is not capable of being a meaningful check on Anthropic. METR is not meaningfully independent, is not sufficiently staffed, and has no authority over Anthropic that cannot be revoked at Anthropic's discretion. Suggesting that embedding METR into Anthropic would be a meaningful check on Anthropic is so suspicious that it looks like an attempt to evade oversight and to sabotage attempts at oversight in general. If Dario does not really mean to suggest that METR could be expected to meaningfully check Anthropic, or he actually believes that METR could check Anthropic in the same way a government regulator can check a bank, something has gone badly wrong in formulating and communicating this policy. Why METR Cannot Check Anthropic You don't have to take my word that they're closely tied to Anthropic, you can take METR's: Also note that some METR staff have strong social ties to employees of AI companies, and METR currently works out of a shared research center (Constellation) which hosts some AI lab staff. [1] I appreciate METR's professionalism and candor, both in general and in this report, because those are not always easy to find. I do think that this somewhat understates the problem: If you required METR employees who dealt with Anthropic to be "independent" the way an accountant doing an audit is legally required to be, METR would probably have to recuse itself from consideration for conflicts of interest. An accountant, for example, would generally be considered conflicted if they had lived with or dated the people they were auditing. It would also be important that the management of the accounting firm, controlling the accountant's career, did not have that sort of history with management in the firm being audited. This proposed arrangement, given the choice of METR, is nothing like embedding a regulatory supervisor at a bank. Bank regulators work for the government and will send bank employees to prison if they do the wrong thing. METR employees would mostly be hanging out with their friends or their boss’ friends at work while drawing non-profit salaries. If METR wants to establish that it isn't hopelessly conflicted here, they should hire an accounting firm to publicly review them for conflicts of interest or to sue me for saying they have them. I think Dario suggesting them as a third-party evaluator analogous to a banking regulator amounts to a malicious lie intended to sabotage regulations and audits, and METR's failure to correct the record makes them complicit in the lie. It seems strange to even call this regulatory capture. They are not at all regulators, and they are not being captured; they were never meaningfully independent, and were never regulators. Dario is simply asserting that they are like regulators. This is so far from even being plausible that it is baffling that anyone would say it. If this arrangement came up in court it would be called incestuous, and if this is the basis for anything like a regulation or a legally binding agreement between companies it will come up in court. This is my first and most serious objection to this suggestion, but there are more. Staffing is also a problem here. METR does not have sufficient staff. Point blank, as a matter of numbers, there are not enough people at METR. METR tells the press that it has about 35 employees, [2] and I don't think more than a tiny fraction of those employees are directly working on model evaluations. I am not entirely sure I could start a poker game with METR's relevant staff. Team sports are right out. Anthropic employs thousands of people and has millions of users. Due to the relevant numbers alone, this is not a serious regulatory proposal. This is a fig leaf, and maybe the plan is to fake it til you make it, to try to hire a ton of people and make this reasonable in the future even if it isn't now, but that is not good enough given the seriousness of the situation. Anthropic is a trillion-dollar business and one of the most important single endeavors on Earth. Even if they were totally independent, METR could not meaningfully check or even record what Anthropic is doing. Next, METR's finances are not meaningfully independent of Anthropic. This applies two ways: first, METR re