Why isn't nicely written prose enough?
Because what you want to do with it is usually not reading. You want to put it in a spreadsheet, drop it into an email, load it into a system, add it up. From flowing prose that is manual work; from a table it is copying.
The format request is the cheapest fix you can make to a prompt: one sentence, and the output becomes usable. That is why The Anatomy of a Working Prompt counts format among the five basic elements.
Level one: you ask for it
In a chat interface this is all there is, and for most tasks it is enough.
State the form, the size and the fields:
Return it as a table. Columns: item, quantity, net price, deadline. If a value is not in the material, put a dash.
If you also want a justification alongside the fields, Chain-of-Thought covers that. It is the last sentence that matters, though. Without it the model invents something for the missing value, because the shape of a table demands that there be one there. This is hallucination in its most insidious form: it does not appear in prose, it appears in a cell, exactly where you are not looking for it.
What works reliably at this level is structure. How many columns, in what order, bullets or paragraphs. What does not is a precise word count, because the model does not know in advance where it will end up when it starts.
Level two: JSON, so a machine can read it
If the result is not for you to read but for another program, you need JSON — a simple machine-readable format in which every piece of data has a name.
You can ask for it in a chat interface too, and it will roughly work. There are two typical problems.
The model talks around it. "Certainly, here is the JSON you asked for:" — and the program that was going to process it falls over. The fix: state explicitly that the answer must be only the JSON, nothing else.
Sometimes it is not valid. A bracket is missing, or it names a field differently than it did yesterday. This is not a rare fault but the natural variability of the mechanism.
That last one improves a lot when temperature is low. If an automated process occasionally returns data in the wrong format, turn this down before you start rewriting the prompt.
Level three: an enforced schema
Through the API there is an option a chat interface does not offer: you can specify the structure in advance, and the system guarantees the answer conforms to it.
Two different things are worth separating, because the names are similar:
- JSON mode — guarantees only that the output is valid JSON. Not that it contains your fields.
- Structured output — you supply the schema, and the answer conforms to it: these fields, these types, this shape.
Of the two, the second is the useful one, and the providers' own recommendation is to use it wherever it is available. In practice the difference is that you do not have to validate and retry afterwards: if the schema is there, the fields will be too.
This is the point where prompting turns into development. But it is worth knowing about even if you will not be building it — because if you tell a developer "sometimes it comes back in the wrong format", this is the answer you will get.
What to specify, in the schema or in the format request
Four things, in this order:
- Field name. Exactly as the other system expects it. Not "price" if it is called
net_priceover there. - Type. Number, text, date, yes/no. For dates, the format too:
YYYY-MM-DD. - Allowed values, if the list is closed. "category: billing | shipping | complaint | other" — without this the model invents a fifth category.
- What to do with a missing value. Empty string,
null, a dash — but say which.
An example that shows all three levels
The task: extracting data from incoming invoices.
Level one, in chat: "Read the invoice number, the issue date, the net total and the payment deadline off this invoice. Return a table. Anything you cannot read clearly, mark [illegible]." Perfect for one or two invoices.
Level two: the same thing asked for in JSON so it can be imported. Works for a few dozen a month, but expect to fix things by hand occasionally.
Level three: a schema, enforced, at low temperature. At several hundred a day this is the only option where you do not have to check everything.
The answer is not decided by the technology but by the volume. That is the fourth question in Can AI Do This Task?, the one-minute method.
What to take away
State the structure, and state what happens with a gap. Those two sentences do most of the work. An enforced schema is more reliable still, but you only need it when enough items pass through the process that checking by hand no longer fits.