Skip to content
aiqbench takeAI Research

aiqbench analysis

This article analyzes the methodological complexities in measuring AI's impact on productivity, specifically addressing how task substitution and shifts in work allocation distort standard efficiency estimates. It draws parallels to economic price index theory to argue that simple time savings on existing tasks underestimate true value uplift while including new AI-enabled tasks overstates pre-AI baselines.
Source coveringMETR
What this isA model's reading of the article and related coverage, written for aiqbench. Not the publisher's words.

Task Substitution and Uplift

METR10 min read

One complication in measuring AI’s effect on productivity is that it has different effects on different tasks, and this causes people to change how they allocate their time between tasks. This makes it more difficult to talk about the effect of AI on overall productivity. We use “old tasks” to mean the set of tasks you’d do in a typical day before AI is available – your average workday in 2021, say. “New tasks” means the set of tasks you’d do in a typical day after AI is available. Not all new tasks necessarily use AI; they’re just the tasks you choose knowing AI is an option. Uplift on old tasks: The factor by which pre-AI time exceeds post-AI time to complete the old tasks. Uplift on new tasks: The factor by which pre-AI time exceeds post-AI time to complete the new tasks. Uplift in value: The factor by which post-AI value exceeds pre-AI value, allowing for reshuffling of tasks between the pre-AI and post-AI cases. In some cases value has a natural definition; in others, it can be operationalized using related definitions discussed more in the accompanying note. This note discusses…

Excerpt shown under the publisher's RSS terms · 192 words · full text and figures are on the publisher's site

Related coverage