What is a token?
The model breaks text into tokens: words, fragments of words, punctuation. An English text runs to roughly as many tokens as it has words, plus a third. Common words get one token each; rare ones split into several.
Why not work in words? Because this way it can handle words it has never seen. "Overengineered" is not in any dictionary it was given, but its pieces are.
And this is where other languages get their invoice. Tokenizers were built overwhelmingly on English text, so they know English words whole and chop everything else into pieces. The same content in Spanish or German typically costs around half again as many tokens as in English; in Hungarian, Finnish or Japanese it can be two or three times as many. This is not theoretical: process a long document and you pay multiples for the same work — and you hit the length limit sooner.
There is an immediately usable trick in this: if the language of the input does not matter, give it to the model in English. You can still ask for the answer in your own language.
What is the context window?
The context window is the largest amount of text the model can see at once. It is not its memory — it is its field of view.
Everything counts towards it:
- the system instruction you never see
- the entire conversation so far
- every uploaded file
- and the answer it is currently writing
Large models today typically work with a window of around one million tokens. That is roughly a hefty novel — or a few hundred pages of company documentation.
What happens when it fills up
You do not get an error message. That is the problem.
Most interfaces quietly start trimming or summarising the beginning of the conversation so the recent part still fits. The model carries on answering — it just no longer remembers what you agreed two hours ago. That is where the feeling comes from that it "got dumber" towards the end of a long session.
There is a subtler effect too: accuracy degrades well before you reach the limit. In a very long text, the model uses information sitting in the middle less reliably than information at the beginning or the end. That is why it pays to put the important instruction at the end of a prompt rather than in the middle.
The logic of the bill
You pay for two separate things:
- Input tokens — what you hand over. Cheaper.
- Output tokens — what it writes. Typically three to five times more expensive.
That difference has two practical consequences.
The cost of a long conversation is not linear, it is quadratic. With every new question, the whole conversation so far passes through again as input. By the twentieth question you have paid not twenty times what the first one cost, but a great deal more.
"Summarise briefly" saves money, "elaborate" spends it. If you are automating something, the length of the output is the main cost driver — not which clever model you picked.
A concrete example
You have a 40-page contract and ten questions about it.
The bad way: upload it and ask the questions one after another in a single conversation. The contract goes through as input with every question — you pay for those 40 pages ten times over, and by the end of the conversation the model no longer recalls exactly what was said at the start.
The good way: ask all ten in one message, as a numbered list. The contract passes through once. If the answers raise follow-up questions, start a fresh conversation for those and hand over only the relevant section.
Same result, a fraction of the cost — and more accurate, because the model sees the whole thing at once.
If you work with the same documents regularly there is a better solution still: instead of handing over the whole body of material with every question, you search out the relevant part first. That is what Fine-tuning, RAG or a Better Prompt? is about.
Two common misconceptions
"A bigger window is always better." No. A large window is a permission, not an obligation. Give it eighty pages when three of them matter and the seventy-seven pages of noise degrade the answer and inflate the bill. A good prompt supplies a small, well-chosen body of material.
"As long as I don't delete the conversation, it remembers." Within the conversation, yes; between conversations, by default no. Several tools have a separate memory feature, but it does not change the model either: it puts notes in front of it at the start — and those eat into the context window too.
What to take away
Two rules will win you most of what there is to win:
- Start a new conversation for a new stretch of work instead of patching onto the old one.
- Hand over only what is needed. The context window is not storage, it is attention — and attention is scarce.