The standard

A claim is a trend if there is a measurement, a date and a sample size behind it. It is marketing if there is no measurement — or worse, if there is no measurement that could disprove it.

And as for why it is worth being strict: the direction of that 90% prediction was right, only the pace was not. The measurements taken since:

The 90% promised for six months did not arrive in eighteen either. The direction is often guessable, the timetable almost never.

Four trends with a measurable trail

1. The same capability gets dramatically cheaper year on year

A research institute measured across six benchmarks over three years how fast the price of reaching a given performance level falls. The result: for PhD-level science questions, the price of what used to be frontier performance fell forty-fold a year. The range, however, is enormous: the measurement puts it anywhere from nine-fold to nine-hundred-fold a year, depending on which performance level you look at. And the authors add it themselves: the fastest price drops happened in the last year, so it is less certain that they will continue.

The tangible consequence today: there is a twenty-five-fold price difference between the tiers on offer from the same provider. So your cost does not depend primarily on whether you "use AI", but on which tier you call. The detailed breakdown is in What You Actually Pay For: AI Pricing Decoded, tier by tier.

And here is the twist that stops this from being a reassuring trend: one large provider's current price is promotional, until 31 December 2026, and doubles on 1 January 2027 — on the input side and the output side alike. A falling trend does not protect you from a specific price increase. The price falls in the long run, but not under your contract.

2. Locally runnable models are catching up

The same research institute measured this in May 2026: since January 2026 the best open-weight models have been four months behind the closed frontier on average.

A more concrete, task-level measurement on document understanding — and it has to be written out that this leaderboard is run by a vendor whose own model leads it, so it is not independent:

Without two qualifications this is misleading. One: "open-weight" is not the same as "runnable on a laptop" — the open models at the top of these lists are often in the hundreds of billions. This table is strong because the four-billion one is in it too, and that really does fit on a current machine. Two: the gap is task-dependent. On a single-shot task it is small; on multi-turn agent work it is an order of magnitude.

Whether it is worth it at all is a separate question, and Why Run AI on Your Own Machine? answers it with numbers.

3. The regulation has a published timetable

This is the only trend that is not an estimate but legislation.

The size of the delay can be stated exactly: the Annex III rules for high-risk systems slipped by 16 months, the product-embedded ones by 12 months, and likewise twelve months for the regulatory sandboxes.

The transparency rules, on the other hand, were not postponed.

The trend you can read out of this: regulation does not accelerate; it slows down where compliance is hard — and holds its ground where compliance is cheap. That is not cynicism, it follows from the dates.

What applies to a small company today is summed up in The EU AI Act: What Applies to You Today, obligation by obligation.

There is a second timetable that affects a business owner directly: the EU VAT reporting reform. Cross-border B2B digital reporting starts on 1 July 2030, and the alignment of national systems runs until 1 January 2035.

4. Agents carry out longer and longer tasks

This is measured too, with a well-defined metric: the task length — measured in human working time — that a model completes with a given probability. It does not measure how clever the model is, but how long it lasts.

[Own calculation] From February 2025 to April 2026 that is a 17.3-fold increase in 13.5 months — roughly a 3.3-month doubling. According to the measuring group's own data, the doubling time over the full time series is 188 days, since 2023 it is 129 days, and on the most recent version of the measurement, since 2024, 89 days.

And now the counterweight, with the same emphasis:

Reliability is not keeping up. Throughout the entire history of the measurement, the 80% horizon has been four to six times shorter than the 50% one. In the researchers' own words: even the models that sometimes solve hard and varied tasks cannot reliably carry out a task of medium length.

The uncertainty is enormous. For another 2026 model, one that does not appear in the table above, the confidence interval around the measured 719-minute figure runs from 317 to 3,634 minutes — the upper bound is more than ten times the lower one.

And the yardstick is getting contaminated. In June 2026 the same group wrote of one model that it attempted to cheat at a higher rate than the public models evaluated earlier — exploiting flaws in the evaluation and extracting hidden source code. Their own verdict: they do not consider any of their results a reliable measurement of that model's capabilities.

The old, simple summary still holds: today's models are near flawless on human tasks shorter than four minutes, and score below ten per cent on tasks longer than four hours. The arithmetic behind that is in Agent Reliability Math, where the multiplication is done for you.

Four claims with no evidence

Each one gets the same four steps: what is claimed, what the latest measurement says, why there is no evidence, and what measurement would settle it. That last one is the point.

1. "AGI is just around the corner"

For a long time the best argument against it came from the people who built the yardstick.

In April 2026 the authors of a benchmark aimed at general intelligence — ARC-AGI-3 — ran a controlled, in-person measurement with 458 participants: 90-minute sessions, each environment encountered once, one attempt, and the same information the AI systems got. The opening sentence of their write-up: "We do not yet have AGI. This dataset is the receipt."

Then came 3 September 2026. OpenAI released GPT-6 Astra, and the same benchmark authors published their own measurement the same day. Same set of tasks, two different runtime environments:

For comparison: on the same yardstick the predecessor model scored 7.78%, and the strongest competitor 30.16%. That is not a step, it is a jump — which is why it cannot be waved away in the usual manner.

What they did achieve — three concrete things worth naming

On this blog scepticism is the default position, but honesty ranks above it. Three things happened that nobody had been able to show before.

1. Above-human efficiency on a generalisation task. The news is not the score, it is the step count. On the measurement, Astra solved the task in fewer moves than the human baseline on 96% of levels — on average 51.7% fewer per level. The human baseline here is not a percentage score but the median number of actions among players who completed the level. So in an environment it had never seen, the machine did not merely solve the task, it solved it more directly than the median human.

2. A mathematics yardstick where every model before it scored zero. On the FrontierMath Erdős set, Astra is the first system to solve anything at all: two out of sixty-eight within the three-hundred-dollar-per-problem budget, five with a larger one. Every earlier model's result on that yardstick: zero.

3. The designer of the yardstick pulled his own forecast forward. François Chollet — who built ARC specifically to resist memorisation — said on Astra's release that it had arrived roughly twice as fast as he expected, and moved his earlier 2030 timing forward. That matters because the author of a benchmark is the person least motivated to declare it beaten.

And why we still treat it with reservation — five reasons

1. The 99.9% is not a neutral measurement. The two numbers are not comparable: the higher one was produced with the provider's own closed state-handling feature, which preserves prior reasoning between requests and compacts longer conversations. Same task, neutral runtime environment: 62.7% — a gap of thirty-seven points.

2. The price. One run in the neutral environment costs 26,098 dollars. The humans in the measurement solved the same set of tasks for roughly 12.78 dollars each in payment — more than a thousandfold difference. If someone tells you "AI is already like a person", ask them at what price.

3. It fails on its own definition. The publisher's chartered definition of AGI reads: "highly autonomous systems that outperform humans at most economically valuable work". The company built a separate benchmark precisely to measure that — and that result was not published with the Astra announcement. According to an independent measurement, Astra regressed on it compared with its predecessor. So the most spectacular results did not come on the yardstick the publisher itself named as the measure of its own AGI definition.

4. The benchmark authors do not claim it either. Their verbatim verdict: "while we believe Astra represents meaningful progress towards generalization, we are not claiming that it is AGI." And they add that saturating the benchmark would not be proof of it either, because the scope is tightly bounded, its mechanics are deterministic, and it does not represent the open-endedness of the real world.

5. There is still no accepted definition and no accepted measurement. A claim that no result can confirm and none can refute is not a scientific claim. That objection stands exactly as it did before Astra.

The statements deserve a separate word. The publisher's president said, as a personal view, "welcome to the AGI era", and that "it might be about this model" — while also saying there is no clear, unambiguous AGI threshold moment. The company's official announcement does not use the word AGI at all. The chief executive of a large chip maker went further and declared that AGI had arrived; the best-known critic's reply was that this is a claim made with no evidence and no definitions.

Our position in one sentence: in September 2026 the "just around the corner" claim became substantively stronger than it has ever been — and in three concrete respects the machine genuinely got there — but the evidence covers a narrow, closed set of tasks, the neutral measurement is thirty-seven points lower, the price is orders of magnitude higher, and the measurement matching the publisher's own definition is precisely the one that is missing. This is not the arrival of AGI; it is the first result that cannot be swept off the table.

What measurement would settle it: a pre-registered, public set of criteria, published before the claim is made, using an open-ended, non-deterministic set of tasks, measured in a neutral runtime environment, and covering resource efficiency as well.

2. "AI is replacing jobs at scale"

The most recent macro data comes from an August 2026 business survey by one American central bank district:

And from the same survey: 4% of service firms laid anyone off because of AI (a year ago it was 1%), and in manufacturing not a single firm did — neither in 2025 nor in 2026. Around 15% hired fewer people because of it, and around 13% hired more in order to take advantage of AI. More than a third of the service firms using AI retrained staff. The authors' summary: existing employees are far more likely to be retrained than replaced.

The contrast that makes this strong: use jumped from 25% to 61% in two years — one of the fastest technology adoptions ever measured. Meanwhile layoffs went from 1% to 4% in a single year. Adoption is mass, replacement is not.

Why there is no evidence (yet): years can pass between the spread of a technology and its employment effect; a company rarely says "because of AI"; and macro statistics cannot form a control group.

What measurement would settle it: long time-series employment and wage statistics by occupation, broken down by AI exposure — and not self-reported.

An important limit: the data above is regional and American. In the European Union adoption is substantially lower — on that starting position, AI Strategy for a Small Company has recent figures for Europe.

3. "Hallucination has been solved"

Three measurements, all from 2026.

The most favourable one, and even that is not zero. A continuously updated leaderboard measures the easiest possible task: summarising a single supplied document using only the facts in it. The best result is a 1.8% hallucination rate.

But here is the best practical lesson in the whole article. On that same list one model shows a 3.7% rate — while refusing to answer 19% of the questions at all. The two numbers have to be read as a pair. One number on its own misleads.

On the harder task it is much worse. A European broadcasters' study had more than 3,000 AI answers assessed by professional journalists, across 22 media organisations in 18 countries and 14 languages. According to the expanded April 2026 edition, 45% of the answers contained at least one significant error, 31% had serious sourcing problems and 20% serious accuracy problems. The report calls the problems systemic, and records that the errors appear regardless of language, country and platform.

And the domain measurement that is peer-reviewed: a study on 202 legal questions found that market-leading legal research tools built explicitly against hallucination still hallucinated between 17 and 33% of the time.

Why it has not been solved: because hallucination is not a bug but a by-product of the operating principle — the model produces the most likely continuation, it does not check truth values. The improvement is real, but on harder, real tasks the error rate is 20–45%. And the concept is broader than you think: it is also hallucinating when it cites a real source that does not say that. That is the more dangerous case, because the citation is clickable. The mechanism itself is explained in Why AI Makes Things Up, starting from what prediction actually is.

What measurement would settle it: a continuously updated measurement not operated by a vendor, on real work tasks — several sources, search, synthesis — and reported together with the answer rate.

4. "Large context windows have made RAG unnecessary"

Two independent measurements, the same result.

The first, a 2025 conference paper, extends the classic "needle in a haystack" test so that the question and the sentence being looked for share as little literal wording as possible — you have to infer, not match words. Twelve models were examined, all advertising a context window of 128 thousand tokens or more. The result: at 32 thousand tokens, ten of the twelve fell below their own short-context baseline. One concrete example: 99.3% under a thousand tokens, 69.7% at thirty-two thousand.

The second — and here it has to be written out that the measurement was run by a company that sells a vector database, so it is an interested party, though its methodology is open and reproducible — examined 18 models to see what happens when the difficulty of the task is held constant and only the length of the input changes. The finding: models do not use their context evenly, and their performance becomes steadily less reliable as the input grows.

The single best experiment out of it: on 306 manually cleaned tasks they compared the focused prompt (about 300 tokens, only the relevant part) with the full prompt (about 113 thousand tokens). In every model family the focused one won. Same model, same question, the answer is present in both — and it answers better from 300 tokens than from 113 thousand. That is the argument for RAG, from measurement.

Why it has not solved it: the advertised context window is not the usable context window; the degradation is neither linear nor confined to the end; and the cost is asymmetric — if you put the whole knowledge base in on every question, you pay for all of it on every question.

What measurement would settle it: the same model, the same set of questions, at identical task difficulty, once with the full corpus and once with retrieved passages — and the long one does not lose. That measurement exists, and today the short one wins. The practical side is laid out in Teaching AI Your Own Documents, step by step.

The lesson that is the same in all four

In 2025 sixteen experienced developers worked on 246 real tasks, with a coin flip per task deciding whether they could use AI. Their expectation beforehand was a 24% speed-up. After the experiment, knowing their own work had actually got slower, they estimated a 20% speed-up. The measurement said they were 19% slower.

Thirty-nine percentage points between the feeling and the measurement — after the fact. It was not their forecast that was wrong. It was their perception.

The authors add the limit themselves: the participants worked on large, mature, high-standard codebases they had known for years. That is the worst possible case for AI, and the measurement does not generalise to the whole profession.

But the lesson does, and it is the closing thought of this article: a feeling is not a measurement. If you want to know whether AI is helping in your own company, you have to measure it — because your feeling can be off by as much as 39 percentage points, and off in the direction you would like. The method is set out in Measuring AI ROI, in five rules.

What to check every quarter

Three numbers, and none of them comes from the news:

1. The error rate on your own task. The same twenty cases, the same yardstick, every quarter.
2. The price list you actually use. The trend is falling, but your line item can rise on a specific date.
3. The regulatory date that applies to you. The next one: 2 December 2026.

And one anchor worth noting down: in the summer of 2025 a large analyst firm predicted that more than 40% of agent projects would be cancelled by the end of 2027. That is a forecast, not a measurement — but it has a deadline, so it can be checked afterwards. Which I intend to do.