How AI Changed Programming

The number everyone quotes

An experiment published in 2023 has turned up in every slide deck since: the group using an AI code completion tool finished 55.8% faster than the control group.

What rarely gets said alongside it: the task was writing one HTTP server in JavaScript, from scratch, as fast as possible. And the authors are employees of the company that sells the tool.

That does not invalidate the measurement. It tells you what it applies to: a self-contained, well-defined, greenfield task where there is no existing codebase, no house style to follow, and no code you are not allowed to break.

The number nobody likes

In July 2025 a stricter experiment was published. Sixteen experienced open-source developers, 246 real issues, with a coin flip per issue deciding whether AI was allowed. The repos average 22,000 stars and over a million lines, and the developers have been working on them for five years on average. The rate was $150 an hour, so motivation would not be in question.

The result: the work took 19% longer with AI.

The authors add the most important limitation themselves: "we do not claim that our developers or repositories represent the majority of software engineering work". This is the worst case for AI: a mature codebase, a high quality bar, and people who know the code by heart. Which is exactly what makes it interesting.

And it is hard to wave away, because of how robust it is: they worked through twenty possible explanatory factors — project size, quality bar, prior AI experience — and the slowdown survived every one of those splits.

The 39 percentage points

Here comes the part that matters more than either measurement.

Before the experiment the developers expected a 24% speed-up. After it — having actually worked more slowly, in their own skin — they still felt they had been 20% faster. The measured figure was a 19% slowdown.

Their forecast was not the thing that was wrong. Their perception was. That 39-percentage-point gap is why nothing in this area can be decided on impressions. What to measure instead is what the measuring the effect post walks through — with marketing examples, but the method is the same.

One important note: in February 2026 the same research group announced that their experimental design had broken down — 30–50% of developers now refuse to do certain tasks without AI, so they pick and choose what they submit. Their position today is that in 2026 AI probably speeds things up, they just do not know by how much. So that 19% slowdown is not a 2026 fact but a piece of early-2025 measurement history — the 39-percentage-point gap, on the other hand, has not gone away.

What the large samples agree on

The largest industry survey asked close to five thousand professionals in 2025. It has four numbers that together tell the whole story:

• 90% use AI in their work;
• more than 80% say their productivity has gone up;
• 30%, on the other hand, have little or no trust in AI-generated code;
• and in the delivery metrics: throughput improves, stability degrades.

That last one is the key sentence. In the report's own framing, AI is an amplifier: it magnifies the strengths of well-run organisations and the dysfunctions of struggling ones.

Another measurement, based on the self-reporting of 400+ companies, shows the same thing in one practical number: the median PR size grew from 44 lines to 72 between July 2025 and June 2026. More code goes through a single review than before — and review capacity has not grown with it.

The technical debt you can measure

There is a code analytics company that examined 623 million code changes between 2023 and 2026. It is not peer-reviewed work, and it does not publish its repo selection criteria — but the orders of magnitude speak.

In one sentence: duplication has grown and the share of refactored code has fallen to a fraction of what it was — developers today copy rather than refactor. Which is no surprise — copying is what AI is fastest at. The bill arrives later.

The J-curve

The most serious modelling of the return was done for a 500-person engineering organisation: 39% first-year return, with a payback period of about eight months.

But the same model also spells out the thing that kills half of all rollouts: a temporary drop in productivity comes first, because of the learning, the increased review burden and the process rework. And in the model the share of failed changes rises from 5% to 6%, which on its own eats several hundred thousand dollars of downtime cost back out of the gain.

The report's most important sentence: the biggest return comes not from the tool, but from whether the organisation adapts the way it works.

"How much code is written by AI?"

The range of answers to this question is instructive in itself.

Stage claims from executives, with no methodology attached: in the autumn of 2024 the CEO of a large technology company said more than a quarter of new code was AI-generated; by April 2026 the same person was saying 75%. The head of a model developer predicted in the spring of 2025 that AI would be writing 90% of all code within six months — that did not happen.

The best measurement available, from the self-reporting of 400+ companies: 27.4% in the first quarter of 2026 and 51.9% in the second. This is self-reported too, and the provider itself flags the risk of bias in both directions.

What is worth taking from this: the number depends on what you call "code written by AI", and the definition is almost never supplied with it.

The labour market

This is where the most reliable data is, from an American payroll database, running to June 2026.

Employment among 22-to-25-year-olds in occupations heavily exposed to AI is roughly 19% below where it would be had it tracked the less exposed occupations. Among experienced workers there is no comparable gap. The mechanism of the adjustment: less hiring, not more firing.

Two important qualifications. The authors themselves stipulate that these are descriptive patterns, not causal estimates. And the effect holds up even when technology companies and computing occupations are taken out — so this is not purely an "AI is taking the junior developer jobs" story, but something broader than that.

A 2026 fashion worth avoiding

Early in 2026 the token consumption leaderboards started to appear: who burns the most tokens in a month, as a productivity measure.

The same research group that runs the largest industry survey rejects this outright as a vanity metric: in their reading, ten times the token spend buys only a marginal increase in throughput. If someone introduces a leaderboard like that where you work, it is not measuring productivity, it is measuring spend.

Common misconceptions

• "It is proven to be 55% faster." On writing a greenfield HTTP server, in the vendor's own measurement. On a mature codebase the answer comes out differently.
• "AI slows developers down." That cannot be claimed either: the original research group withdrew its own measurement design, and today says it probably speeds things up.
• "More code = more value." The median PR went from 44 to 72 lines and duplication rose by 81% — the quantity grew, the maintainability did not.
• "The return arrives as soon as you roll it out." The modelling says productivity falls first. Anyone who has not planned for that switches it off at the bottom of the dip.
• "Everyone trusts it, since 90% are using it." In that same survey, 30% of professionals have little or no trust in the generated code.

What to take away

Two things have definitely changed: more code is produced, and it moves through the pipe faster — while stability degrades and duplication grows. The gain is not in the tool but in whether the team adapts its process: review, testing, approval. And your own feeling is the worst measuring instrument available.

Which working mode fits which job in practice is covered by the three modes of AI coding post, mode by mode; the concrete failure patterns to look for in review are laid out in the reviewing generated code post, with examples.