The sloppy research fuelling fears of superhuman AI

Published on October 5, 2026 7:51 PM GMT If there is one piece of evidence used to support fears of superhuman AI, it’s this graph: The graph is supposed to show exponential improvement in large language models (LLMs) on a variety of tasks, especially software engineering. The graph compares two things: 1) how long it takes humans to do certain tasks and 2) whether LLMs can do those same tasks and succeed at least 50% of the time. A reasonable question to ask is: how do the people who made the graph know how long it takes human to do those tasks? The answer is: they tried to measure some tasks (with a flawed methodology that rewarded people for taking longer), but for a lot of the tasks, they just guessed. In fact, they said that for most of the “longer tasks” (what exactly “longer” means, they don’t say), they just guessed how long humans take to complete them. In other words, the “data” here is not data. It’s not measured. It isn’t obtained empirically in any way. It’s just made up, pulled out of thin air. When you first saw this graph, did you assume that it was a graph based on real data? That is to say, data that was somehow objectively measured? Most people do. It’s easy to miss the extent that guessing is involved if you don’t look carefully. In fact, the organization that made the graph, METR, didn’t disclose until ten months after the graph’s release that the human data for most of the longer tasks was guessed rather than measured. On Twitter, METR’s initial announcement of the graph (and the underlying model) omits any mention that some of the data is guessed. Instead, it gives readers the misleading impression that it was all measured: At a high level, our method is simple: 1. We ask both skilled humans and AI systems to attempt tasks in similar conditions. 2. We measure how long the humans take. 3. We then measure how AI success rates vary depending on how long the humans took to do those tasks. The Twitter thread again says that they “measure human and AI performance” and refers to “how long it took humans to do each task”, rather than how long it was guessed humans might take to do the tasks. If you dig into the “Methodological Details” section on METR’s blog post about the graph (or read their non-peer-reviewed paper), you will find disclosures that they didn’t actually measure human performance on all tasks. But the disclosures aren’t prominent enough for most casual readers to notice. In the Twitter announcement — which has apparently been viewed millions of times — there’s no disclosure at all. There are two obvious reasons why guessing rather than measuring is problematic, which is why it’s not accepted in scientific research. First, guesses are often wrong. The whole point of science is to learn new information through experiment and observation, rather than just repeating what we think we already know. Second, guesses are susceptible to bias and scientific misconduct. If “data” can be invented by the people doing research, they might intentionally or unintentionally invent data that suits them, perhaps because it fits with the conclusions they want or expect to find. It’s easy to explain this problem with METR’s graph — the graph’s reliance on guesswork rather than empirical measurement — and why it’s problematic. But there are many other problems with the graph that are harder to explain, which may be even worse than the problem I just described. Research writer Nathan Witkin explains many of the graph’s problems here. Cognitive scientist Gary Marcus and computer scientist Ernest Davis tackle a few others here. The upshot is that the METR graph is, on the whole, a mess. It’s not based on what would typically be considered acceptable scientific research. METR’s paper would not likely pass peer review at a credible scientific journal, if it were submitted. What’s the big picture, here? The METR graph is by far the most commonly cited piece of evidence to support the notion that we should fear superhuman AI. In fact, it’s often the only piece of evidence cited. There is little else out there that even attempts to substantiate superhuman AI fears through a rigorous, empirical, scientific methodology. This should give us pause. How can people fear superhuman AI based primarily on just one piece of evidence that’s unacceptably bad? AI 2027 A somewhat lesser-known but still widely publicized and shared piece of “evidence” for near-term superhuman AI is a website called AI 2027. Remarkably, AI 2027 follows the same troubling pattern as the METR graph. Like the METR graph, it has multiple profound methodological problems that render it unacceptable by typical scientific standards. Some of these problems are detailed here by the philosopher David Thorstad. Also like the METR graph, AI 2027 presents a scientific-looking “result” that largely depends on numbers that were guessed rather than measured. And like the METR graph, the amount of guesswork involved in AI 2027 was only fully disclosed