The number everything else follows from
In early 2025 a research group ran a rigorous experiment: 16 experienced developers, 246 real tasks, with a coin flip per task deciding whether AI was allowed. Beforehand the participants expected a 24% speed-up. The measurement showed a 19% slowdown. And more important still: after the experiment, having lived through the slowdown, they still believed they had been 20% faster.
There is one thing you must not take from this: that "AI slows you down". The measurement was run with early-2025 tools, and in February 2026 the same team announced that their design had broken down — 30–50% of developers now refuse to do certain tasks without AI, so they pick and choose what they submit. Their position today is that in 2026 AI probably speeds things up, they just do not know by how much.
What does hold, and it is this post's thesis: self-report and measurement come apart systematically, in a predictable direction. The same team put a number on that too: in their earlier measurement, people overestimated how much AI had helped them by 39 percentage points. Thirty-nine percentage points — far bigger than the effect you want to decide about.
The same thing in marketing, with numbers
You do not have to lean on an analogy from software. There is marketing data for it, and it shows exactly the same shape.
In a survey of 1,015 B2B marketers published in the autumn of 2025, this is what the staircase looks like among those who use AI for content production:
| What was asked | Share reporting an improvement |
|---|---|
| Productivity | 87% |
| Operational efficiency | 80% |
| Content quality | 58% |
| Content performance | 39% |
Of the same respondents, 34% said nothing had changed and 5% said it had got worse.
Another survey, of 1,005 marketers, produced the same thing: 83.8% report a productivity gain and savings of more than five hours a week, but only 25.6% say the AI content also performs better. Two surveys, two separate periods, two samples — the same shape: roughly 85% feel an improvement, 25–40% see a performance improvement.
And here is the twist: that 87% is the same kind of self-report as the one the experiment above measured to be 39 percentage points too generous. So the gap between the 87% and the 39% reporting better performance probably understates the real one.
The four metrics
The field has no peer-reviewed measurement framework of its own. There is a well-known one from software engineering, though, and it says exactly what is needed here: never look at a single number, and activity on its own is not productivity. In its own words, activity metrics "should never be used on their own" — not to reward and not to punish — and it is worth measuring at least three different dimensions at the same time.
Translated to marketing, these are the four:
(a) LEAD TIME / COST PER PIECE -> how much time and money per piece
With a stopwatch, not a questionnaire. This is the one that
gut feel distorts the most.
(b) OUTPUT VOLUME -> how many pieces got produced
WORTHLESS ON ITS OWN. Only readable together with (a) and (c).
(c) BUSINESS METRIC -> conversion, revenue, enquiries
This decides whether it was worth it. The other three explain.
(d) SCRAP AND REWORK -> how much had to be fixed, binned,
sent back. The hidden cost side almost nobody measures.
ALWAYS ALL FOUR. If you only look at (a) and (b), every AI tool
will look like a success.
Why (c) decides: five applications, two zeros
The best evidence that the business metric can be measured is a series of randomised field experiments run on a large e-commerce platform. Same company, same year, same technology, five different marketing AI applications — and two out of the five moved nothing at all:
| Application | Effect on sales |
|---|---|
| Pre-sales chatbot | +16.3% |
| Search query refinement | +2.93% |
| Product description generation | +2.05% |
| Marketing push message | +1.6% — not significant |
| Ad headline generation | no measurable effect |
Anyone not measuring books all five as a win, because all five "got faster". The chatbot and the ad headline end up in the same row, even though one of them moved sales by more than sixteen percent and the other by nothing at all.
Why you cannot A/B test a tool rollout
This part hurts, but it has to be said.
About the developer experiment above, the researchers wrote themselves that the 246 tasks were "just barely enough" to reject the null hypothesis. That took 16 paid developers, random assignment per task, and eighty-four hours of screen-recording analysis. A four-person marketing team has a fraction of that.
Let us do the maths. If the spread of your lead times is the usual marketing kind — a newsletter and a pillar article differ by orders of magnitude — then detecting a 20% speed-up takes 97 pieces per group, so 194 in total. At twelve pieces a month, that is sixteen months. A 10% speed-up takes 870 pieces: six years. By then you will have swapped the tool twice.
On the business metric it is worse. At a 2% baseline conversion rate, detecting a 20% relative lift takes 42,217 visitors. At two thousand visitors a month, that is twenty-one months. The detailed sample-size table sits in the testing landing page variants post, but the lesson there is the same.
The right answer is not stricter statistics. It is the before-and-after comparison, with identical definitions, against a baseline written down in advance — and with the knowledge that this is not causal proof, only the best available.
The cost side that rarely gets added up
The minimum tool basket of a four-person marketing team at September 2026 list prices — a language model for four people, image generation, a design tool for two people, one SEO and visibility tracking package — comes to $311 a month, roughly 1.17 million forints a year. Two months of a junior colleague's payroll cost. On top of that come two items that appear on no invoice.
The review time. If every piece needs twenty minutes more checking and twelve pieces get made a month, that is four hours. At an internal hourly cost of eight thousand forints, that is 32 thousand forints a month — more than image generation and the design tool put together.
The fixing. According to a study of roughly a thousand US office workers, between one in ten and one in five incoming pieces of work is "shiny but empty": it looks fine, but it hands the real work on to whoever receives it. Fixing one takes close to two hours, and the cost is estimated at $186 per person per month. For four people that is 233 thousand forints a month — two and a half times the entire tool basket.
That is metric (d). Not a "soft" consideration: if you only measure one side, you get the sign wrong as well.
What people say about ROI — and why you should believe none of it
In the summer of 2025 a report came out that has been quoted everywhere since: "95% of organisations get zero return". What does not get quoted alongside it: it is built from 52 interviews and 153 questionnaires collected at conferences, and the report itself notes that its data rests "on individual interviews rather than formal corporate disclosure" — that is, on the very self-report this post is criticising.
In the same quarter, another survey of 800+ corporate decision-makers found that three quarters of them report a positive return.
Ninety-five percent failure and seventy-five percent success, to the same question, in the same year. When two serious surveys give each other's opposite, the problem is not in the AI, it is in the measurement.
The most defensible number comes from an August 2026 survey with 1,719 respondents across 97 countries: 37% of companies can show any operational earnings effect at all — and that share has not moved in a year. Not 95% failure, not 75% success. Roughly one in three.
Where the countries stand
According to 2025 data from the European statistical office, AI use among companies with more than ten employees varies fourfold across member states: the EU average is 19.95%, Denmark is at 42%, Romania at 5.2%. The figure is worth knowing because it tells you what you are early or late against — and because the marketing share behaves differently from the overall picture. Open the one that applies to you:
Hungary — half the EU average, but those who use it use it for marketing
10.4% of Hungarian companies with more than ten employees use any AI technology at all, against the EU average of 19.95% — roughly half of it. That is 22nd among the 27 member states. For marketing or sales, 3.6% of all companies.
Among those who use it at all, though, marketing is the number one terrain: 34.7% of AI-using Hungarian companies also use it for marketing. Worth knowing that this figure is exactly the EU average (34.7%) — so it is not a Hungarian peculiarity. The Hungarian peculiarity is how few use AI at all, not what they use it for.
A Hungarian SME survey — small sample, from a vendor, no methodology — says the opposite: that AI is used at 73% of Hungarian businesses, though only about a third of them pay for it at company level. The two numbers do not contradict each other: one measures whether somebody at the company uses it, the other whether the company does. The difference is shadow AI itself. If you cannot even see the cost, you are certainly not measuring the effect.
Spain — crossed above the EU average in a single year
20.27% of Spanish companies with more than ten employees use AI technology: 13th out of 27, just above the EU average. Almost double the Hungarian rate.
The turn is recent: in 2024 Spain was still below the EU average (11.31% against an average of 13.48%), and it nearly doubled in a year, +8.96 percentage points — while the EU moved +6.47 and Hungary +2.96. By size: 17.2% at 10–49 employees, 30.1% at 50–249, 57.5% above 250.
The marketing share, however, goes the other way: only 29.5% of AI-using Spanish companies put it into marketing, against the EU average of 34.7%. So more Spanish companies use AI, but proportionally fewer of them for marketing — there, administration and business management is almost as large a share (28.6%).
One rarity you will not find elsewhere: Spain is one of only two member states that also report figures for companies with fewer than ten employees. There the rate is 10.1% — meaning a Spanish micro-business sits roughly where a Hungarian mid-sized company does.
The three metrics to drop
All three make the same mistake: they measure input or activity, not output.
- "How many prompts went out." In the experiment cited above, the people using AI spent more time prompting and reviewing the result — while getting slower. There the prompt count correlated with the worse outcome.
- "How many words got written." This is exactly what the researchers object to: output can be more verbose with the same substance, and a task can be sliced up without the total work changing at all.
- "How many people use it." In a survey of nearly five thousand software professionals, 90% use AI at work and more than 80% feel a productivity gain — while in the same survey delivery stability gets worse. In the marketing survey above, 95% use AI but 39% see a performance improvement. Adoption and effect are two separate quantities.
What to write down before you switch it on
1. BASELINE, IN WRITING, BEFORE the rollout.
For all four metrics, from the last 3 months.
If you try to reconstruct it afterwards, your own memory
will supply it - and we know what that is worth.
2. A DEFINITION for every metric, in one sentence.
"Lead time = from sending the brief to approval."
If you count it differently in six months, there is no comparison.
3. TIME FRAME: at least two quarters.
The first months systematically give a worse picture,
because the learning and the process rebuild show up as
a cost, but do not show up as an investment.
4. DECISION RULE, up front.
"If (c) does not move in two quarters, we drop it."
Nobody makes that call after the fact.
5. THE FULL COST: subscription + review time + fixing.
Not just the invoice.
Common misconceptions
- "We have noticeably got faster." In the most precise measurement available, the feeling was off by 39 percentage points. A feeling is not data.
- "AI actually slows you down." You cannot claim that either: the original research team withdrew its own measurement design, and today says it probably speeds things up.
- "95% of pilots fail." 52 interviews and 153 questionnaires, on the report's own account not from formal corporate data. The defensible number: roughly one company in three can show an earnings effect.
- "We will just A/B test it." At twelve pieces a month, detecting a 20% speed-up takes sixteen months. For a small team that is not a question of laziness but of sample size.
- "Only the time saved counts." For a four-person team, the cost of the fixing is estimated at two and a half times the entire tool basket.
What to take away
AI's effect is not something to feel, it is something to measure — and with four metrics, because one of them always lies. The two almost nobody looks at are exactly the two that matter most: the real business metric and the time that goes into fixing. The baseline has to be written down before the rollout, because afterwards all you have left is your memory — and we now know what that is worth.
What is worth measuring at all inside the AI search engines is what the measuring your brand's visibility post covers; and for the big picture, what actually changed, read the balance sheet on AI in marketing post.