analiza aiqbench
This note analyzes how Anthropic's coding agents have increased researcher output by more than 2x, using economic models to adjust for the fact that coding is only part of a researcher's job. The analysis highlights significant uncertainties regarding AI-generated code verbosity and the distinction between low-stakes versus high-stakes development tasks. Ultimately, it suggests that while coding efficiency has surged, total effective research uplift depends heavily on how these tools are applied across non-coding responsibilities.
Because 8 ≈ e², Anthropic's researcher uplift is plausibly >2x

Notes are unpolished research updates directly from METR researchers, and have gone through less review than our research posts. They do not necessarily reflect the views of METR as a whole. Note: the modeling assumptions and conclusion are Thomas Kwa’s opinion, and others at METR disagree.1 Also, the math was checked by Claude but not a second human. Anthropic’s RSI blog post reported that in Q2 2026, Anthropic contributors merged 8× as much code per day as in the 2021-2024 period. What does this imply about the factor by which a researcher’s total effective output increased — the (serial) researcher uplift2? Of course, 8× more code doesn’t mean 8× more research, as coding is only part of the job. However, if we assume each line of code (LoC) has equivalent quality and verbosity to pre-2025 code and make standard economic modeling assumptions, we can conclude a surprising amount: all models predict researcher uplift at Anthropic from coding agents alone is >2×. (Researcher uplift could be even higher, because these numbers assume no uplift on non-code tasks.) Cobb-Douglas predicts that if pre-AI time spent coding is \(\beta=50\%\) and code output increases by a factor \(M=8\), then researcher uplift is \(M^\beta =…
Extras afișat conform termenilor RSS ai publicației · 200 cuvinte · textul integral și figurile sunt pe site-ul publicației
Subiecte conexe

Measuring the Self-Reported Impact of Early-2026 AI on Technical Worker Productivity
METR · 11 mai 2026
Debating RSI, the US-China Gap, and Jaggedness with JS Denain of Epoch AI
Interconnects · 22 sept. 2026

Research Note: Filtering Subversion-Relevant Information From Pretraining Data Is Feasible
LessWrong · 5 oct. 2026

AXRP Episode 50 - Eli Lifland on AI 2027
LessWrong · 3 aug. 2026