You Feel Faster. Are You?

METR ran a randomized controlled study with experienced open-source developers. Half used AI coding tools. Half didn’t. The AI group took 19% longer. When those same developers were asked to estimate their own speedup afterward, they said 20% faster.

That gap — the distance between how fast you feel and how fast you actually moved — is where most AI productivity arguments live.

McKinsey says 46% time savings. DX reports hours saved per week. Both numbers come from surveys and self-reporting — they measure vibes at scale, not task completion time. The METR study measured actual task completion time under controlled conditions, with complex tasks, on real codebases.

Here’s what I notice in daily use: AI tools are genuinely fast at the parts of coding that aren’t the bottleneck. Boilerplate, scaffolding, “write a function that does X” — quick. But the work that actually takes time is understanding why something is broken, making a judgment call about architecture, or figuring out what the right question even is. On that kind of work, the AI-generated context you’re now holding in your head might slow you down more than the autocomplete speeds you up.

The problem is that nobody’s categorizing their team’s work before claiming productivity wins. Routine tasks and complex tasks are different buckets. Feeling faster in one doesn’t tell you anything about the other.

Most companies are running a measurement program that only captures whether developers are happy with their tools. That’s a satisfaction survey with extra steps.