Two entirely different costs
Training — building the model. A one-off operation that runs for months and is staggeringly expensive. Tens of thousands of graphics cards run for weeks, and at the end one file comes out: the set of weights. For a frontier model this is today in the order of several hundred million dollars, and more at the very top.
Inference — using the finished model. You ask, it answers. This runs from a fraction of a cent to a few cents per question.
The ratio between them is extraordinary: for the price of training a single frontier model you could ask a finished one hundreds of millions of questions.
Why does this matter to you?
Because the entire logic of the AI market follows from it — and so does your decision.
You do not pay the cost of training. A handful of very large companies pay it, then sell access. That is why the result of a several-hundred-million-dollar development is available to you for twenty euros a month.
The cost of use, on the other hand, is yours, and it scales. One person asking twenty questions a day is negligible. A customer service system handling twenty thousand a day is a budget line.
This inverts the reflex we bring from conventional software. There, development is expensive and running is cheap. Here running is the recurring item — and so the main question is not "is it worth introducing", but "what does it cost per month once people actually use it".
What is in the price, and what you cannot see
Output tokens are typically three to five times more expensive than input tokens. The reason is in the mechanics: the model processes the input in one pass, but produces the output one token at a time.
There is one item that surprises many people: thinking tokens count as output tokens too. If the model reasons at length to itself before answering, you never see that reasoning, but you pay for it. That is how two models with similar list prices can differ dramatically in actual cost. Reasoning Mode covers it.
Where are prices heading?
In two opposite directions at once, and this is worth saying explicitly, because the received wisdom only knows one half of it.
Down, at the bottom of the field. The price of a given level of capability falls drastically year on year. What only the most expensive model could do two years ago is now handled by a smaller model running at a fraction of the price. This is one of the most reliable trends in the market.
Up, at the top. The list price of frontier models, however, is not falling — in some cases it has risen. Vendors charge more for the most advanced capability, and the heavy long-thinking modes run on separate, higher tariffs.
The practical conclusion: "AI keeps getting cheaper" is only true if you do not insist on the strongest model every time. If you do, your bill will not shrink.
A calculation worth doing
Before you roll anything out, estimate these four numbers:
- How many requests per day?
- How large is the input on average? (An uploaded document is input too.)
- How large is the output on average?
- Does this need the strongest model, or will a smaller one do?
The fourth question is the most important, and most people skip it. The majority of tasks — categorising, summarising, reformatting, extracting data — do not need a flagship model. A smaller, faster model gives the same result for a fraction of the cost, and answers quicker as well.
A good practice: start with the strongest model while the solution is still taking shape. Once it works, run the same prompt on a cheaper one across twenty real cases. If the result holds up, switch. That single step often removes eighty per cent of the cost.
What does this mean for building your own?
The idea of "let's build our own" comes up regularly. It is worth being clear about what that means:
- Training your own model from scratch — effectively out of the question. This is the several hundred million dollars.
- Further training an existing model (fine-tuning) — realistic, but not for what most people imagine. Fine-tuning, RAG or a Better Prompt? covers that.
- Running a finished model on your own servers — realistic, and here the cost of inference is hardware rental rather than a token bill. For data protection reasons this is often the only viable route; Open-weight and Closed Models works through the trade-off.
What to take away
Training is the vendor's problem. Inference is yours — and it is payable as many times as the thing gets used. So the return on an AI project is not decided when you introduce it, but when you decide which size of model to point at which task.