A growing body of evidence suggests that larger AI models may not actually make software developers faster, despite the industry's focus on benchmark scores and parameter counts. Research from the Model Evaluation & Threat Research (METR) group found that experienced open-source developers working on their own repositories were about 19% slower on average when using AI tools for the tasks studied. The finding challenges the common assumption that AI always boosts productivity.
The Benchmark Gap
Every few months, companies announce new models boasting "100 billion parameters" or "state-of-the-art coding benchmark" scores. But writing production software is fundamentally different from solving benchmark problems. Real development involves understanding requirements, debugging edge cases, making trade-offs, reading legacy code, and communicating with teammates—tasks that benchmarks don't capture. While benchmarks measure capability, they don't always translate to real-world productivity gains.
Activity vs. Productivity
The METR study highlights a key issue: developers often mistake activity for productivity. When using AI, they spend more time prompting the model, waiting for responses, reading generated code, double-checking its correctness, and refactoring what the AI produced. This busywork can offset any speed gains from code generation. As management thinker Peter Drucker put it, "What gets measured gets managed." If teams only measure how quickly AI generates code, they ignore the time spent reviewing, validating, and maintaining that code.
The Developer Still Matters
Larger models are more capable, but they don't automatically make someone a faster developer. A skilled engineer using a lightweight model with clear context and good engineering practices can often outperform someone using the latest flagship model without fully understanding what they're building. AI should assist thinking, not replace it. When something breaks in production, developers need to understand their own architecture, decisions, and codebase—not just rely on AI to explain everything.
Ultimately, AI is a tool, not a magic wand. Whether it makes you faster depends less on the model and more on how you use it. The developers who benefit most are those who think critically, understand their systems, and use AI to eliminate repetitive work rather than replace engineering judgment.