Reliable Summaries of Meetings, Reports and Threads

What the model leaves out, not what it makes up

Summarisation is one of the most measurable AI tasks, because the result can be held against the source. And the measurements consistently say the same thing: the dominant failure type is not hallucination but the missing fact. One study puts it in exactly those words: "missing facts are the dominant source of error".

The numbers bear it out. Summarising legal texts, the best models returned 67 to 71 percent of the arguments, while human reference summaries stood at 97 to 98 percent. No false claims went into the text — a third of the arguments simply were not there, and what remained read perfectly plausibly.

This is dangerous in practice because omission cannot be spotted by reading. An invented number catches your eye if you know the subject. A decision that goes unmentioned or a forgotten deadline leaves no trace: the summary is round, readable and short — exactly what a good summary looks like.

The middle, where the point disappears

There is an effect first measured in 2023 that has held up since. Give a model several documents, and when the information sought sits at the beginning, the hit rate was 75.8 percent; at the end, 63.2; in the middle, 53.8. That was with twenty documents. With thirty, the middle value fell to 50.5.

The startling comparison comes from the same measurement: the same model, given no documents at all and working purely from its own knowledge, scored 56.1 percent. So when the correct answer was right in front of it but in the middle of the text, it did worse than if it had been given nothing. A longer context does not solve this: a 2026 analysis argues the effect is not a defect but a consequence of two competing pressures during training — so a bigger window will not remove it. What matters to you is that a decision made in the middle of a thread is the most exposed point there is.

Four failure types that meeting summaries produce

Across twenty real meeting transcripts and three thousand question-answer pairs, more than a thousand ungrounded statements were collected. All four of the most common types show the same thing: the model gets the words right and the social meaning wrong.

• The confident paraphrase — 27 percent of failures. The nuance of what was said disappears: "I still need to look at that" becomes "will review and report back" in the summary. Not a lie, just firmer than the original.
• The unjustified quantifier — 17 percent. "Most participants agreed", "several people raised it." Nothing in the transcript supports the most or the several. This is the most insidious of the four, because it looks like a decision.
• Attribution to the wrong speaker — 16 percent. A proposal moves from the person who made it to the person who spoke next. In meeting notes this has personal consequences.
• The invented action item. The model writes in a task nobody took on. Notably, this failure type is characteristic of the more advanced models — the weaker ones try less hard to round the story off.

The overall picture: the study found the failures do not come from factual recall but from the model misreading the intent of the conversation. That is why picking a better model does not fix it — the failure rate drops, but the type stays.

The recording that arrives already wrong

If your input is a transcript from a recording, the trouble starts one layer earlier. In a study covering 437 speakers, roughly 1.4 percent of transcriptions contained an entirely invented sentence — not a mishearing, but text conjured out of nothing.

The mechanism is the important part: invention correlates with silence. Recordings with more non-speech stretches produced more hallucination. Another measurement pushed this to the limit: on pure environmental noise, with not a single word spoken, one popular transcription model produced text for 99.97 percent of segments. The practical lesson is simple: pauses, typing, corridor noise and people talking over each other are where invented sentences are born — and the summary then receives them as fact.

The recipe

Five steps. The third is the one most people skip, and it carries half the errors.

1. Specify the structure in advance. The request should not be "summarise this" but named sections: decisions, actions, open questions, figures mentioned. Here structured output is not a convenience but the only thing that makes omission visible.
2. Require an explicit "not stated". If a section is empty, that has to be written out. Without it the model quietly drops the section, and you cannot tell whether something is missing or there genuinely was nothing.
3. Ask for a quote beside every claim. A short verbatim fragment from the transcript. This step alone catches the confident paraphrase and the wrong-speaker attribution, because the quote is either there or it is not.
4. Chunk long material — but do the merging yourself. Chunking the input is correct, and processing long documents covers it in detail. Having the model merge the chunks, however, produced worse results in measurements than a single plain request.
5. Ask separately about the middle. A second, short request: "what was discussed in the middle third of the conversation?" This is the fastest antidote to the middle disappearing.

The prompt that punishes omission

This request contains the first three steps of the recipe. Swap the sections for your own.

Summarise the transcript below. Include ONLY what was actually said.

Sections, all of which must be filled in:
1. DECISIONS — what was settled definitively
2. ACTIONS — what, who, by when
3. OPEN QUESTIONS — raised but not closed
4. NUMBERS AND DATES — every figure mentioned, in one place

Rules:
- beside every point, put in quotation marks the fragment of at most 15 words
  it follows from. If there is no such fragment, the point cannot go in.
- if nothing belongs in a section, write: [none]
- if an action is missing an owner or a deadline, write: [not stated]
- do not write "several", "the majority", "general agreement" — name who said it.
  If that cannot be established, write: [not identifiable]
- do not draw conclusions, and do not propose anything

The check that takes two minutes

A summary is not to be read but attacked. Four questions, all quick to answer.

• Is every number there? Search the transcript for digits and check each one appears in the summary's fourth section. This is the fastest check and the one that catches most.
• Does it say "several" or "most"? If so, that is almost certainly ungrounded — by measurement, one sixth of all failures.
• Do the names match the sentences? Two random points traced back are enough. Wrong-speaker attribution rarely travels alone.
• What is missing from the middle? Open the middle third of the transcript and read the section headings or first sentences. If something there looks important and is not in the summary, you have found the error.

What not to do

• Do not ask for a short summary first. The pressure to be brief is precisely what strengthens omission. Ask for a complete one with sections, then compress — or compress it yourself.
• Do not let the model merge the chunks. In summarisation, hierarchical merging and incremental updating produced worse results than a plain request in measurements, and they hide the error as well.
• Do not accept the action list unchecked. The invented task belongs to the more advanced models, and it is exactly the section people start working from. For checking numbers and names, the four-step check sets out a detailed method.

What to take away

With summaries the main risk is omission, not invention, and the omitted part is in the middle. Give it sections, make "not stated" mandatory, ask for a quote beside every claim, and ask separately about the middle third of the conversation. Then do not read the finished summary — attack it with four questions. Two minutes, and it catches most of what went wrong.