Skip to Content

inference

Running the finished model: the step where you ask and get an answer.
September 13, 2026 by
inference
Krisztian Eyssen, Krisztian Eyssen

A model's life has two phases. Training is the one-off, enormous investment out of which the model itself comes. Inference is everything after that: every single question anyone puts to it. Several orders of magnitude separate the two — building it costs hundreds of millions of dollars, answering one question costs a fraction of a cent.

That gap decides what is worth buying and what is worth building. The price of inference is public and measured in tokens, and there is a twenty-five-fold spread between tiers at the same provider: that is the gap between the most expensive and the cheapest tier on offer. Your cost therefore depends less on whether you use AI at all than on which tier you call.

The most common misconception is that inference is "free once you have the model". It is not: every request consumes electricity, hardware and time, and when you run things locally this is exactly the item that does not disappear — it only moves onto your own power bill.

The economics of the two phases is worked through in the cost of training and inference, a separate article. The directly related terms: training, token and local LLM.

inference
Krisztian Eyssen, Krisztian Eyssen September 13, 2026
Share this post
Archive