What was this, and why did it work?
If you ask a model to work the problem through first and only then give the answer, it gets more accurate on complex questions. That is chain-of-thought.
The reason is in the mechanics. The model lays down tokens one at a time, always choosing on the basis of the text so far. If it writes out the working first, then when it comes to the conclusion it can lean on its own earlier sentences. It did not get smarter — it just laid down more footholds for itself.
What changed
This is now built in. Most current models have a thinking mode that does exactly this by itself before answering — that is the subject of Reasoning Mode, a separate article.
From which follows what the providers' own guidance now states outright: with thinking switched on, a hand-written "think step by step" is largely redundant. The model does it anyway.
In fact the advice goes further than that: a general request ("think it through thoroughly") often produces a better result than a prescribed sequence of steps. If you dictate the train of thought, you force the model down a route it would not have chosen — and one the task may not call for.
What makes things worse today
Three habits carried over from the era of older models, and all three now do harm.
The prescribed step list. "First list the facts, second analyse them, third draw a conclusion." If the task is not shaped like that, the model forces the form and the substance gets lost.
Repeated verification. "Check your answer, then check it again." This helped on older models. On current ones it leads to over-verification: it deliberates longer, burns more thinking tokens, and occasionally rewrites a good answer into a worse one.
Asking for working on a simple task. On a categorisation or a rewrite, the working produces nothing but noise and cost. There the format request is what helps, not the train of thought.
When it is still needed
Four cases survive where manual chain-of-thought brings real benefit:
- When thinking is switched off. On cheaper or smaller models, and at low effort settings, the old trick works exactly as it used to.
- When you need to see the working. Built-in thinking stays largely hidden. If you need the explanation — because you have to check it, or because it is for the reader — you have to ask for it separately.
- When the task has a prescribed method. For a checklist or a statutory test, the steps are not directing the thinking; they are part of the task.
- When the model keeps omitting the same thing. Here you do not prescribe the whole train of thought, you add the one missing consideration.
The distinction that trips people up
Two different things go by the same name:
Making it think — so the model works harder. Today that is handled by a setting, not by the prompt.
Asking for the working — so the justification appears in the answer. That is a format request, and it is still your job. "For every claim, state which paragraph you based it on" is not making it think, it is making it checkable.
Confusing the two is why people write needlessly long prompts: they are trying to induce thinking when all they need to do is shape the output.
What it does not solve
It does not eliminate hallucination. With nothing to go on, longer working gives you a better-constructed mistake. Supplying the material is worth more — see Why AI Makes Things Up for the detail.
It does not fix a bad prompt. If the instruction is missing information, the model will simply deliberate longer over the same gap.
A five-minute test
Take a typical, more complex task from your work and run it three ways: with a manual step list, with a plain "think it through thoroughly", and with no special instruction at all.
On most current models the second or third wins. If the first wins for you, that usually means the task genuinely has a prescribed method — in which case you did not write a prompt, you wrote a procedure, and its place is the system prompt, not the daily request.
What to take away
"Think step by step" is not bad advice, it is out of date: the model does it by itself now. What is worth asking for instead is not the thinking but the visible justification — because that you still do not get automatically, and that is what you need in order to check the answer.