Agent Memory: What to Store and What to Forget

What memory is, and what it is not

The most common misunderstanding is that a long context window is memory. It is not. The context window is what fits in front of the model within one run; memory is what survives between runs. That difference is laid out in the tokens and context window post in full.

The established split by scope gives you two:

• Short-term memory — tied to a single session. The thread of the conversation, preserved with checkpoints so that it can be resumed at any time.
• Long-term memory — spanning sessions and conversations, organised into namespaces, reachable at any time from any thread.

By type you get three:

This trio is useful because all three have a different correct expiry time. A semantic fact goes stale (the job title, the company name, the preference changes). An episodic memory does not go stale, it becomes irrelevant. And a procedural rule holds until the workflow changes — and then it has to be swapped out all at once, not left to fade.

When should the agent write?

There are two options, and this is a design decision, not a detail:

• During the run (on the "hot path"): whatever the agent learned is available immediately. In exchange, latency goes up and the system gets more complicated.
• In the background, asynchronously: no latency, but you have to decide when and how often the summarisation step runs.

The practical compromise is usually that fact writing goes on the hot path (short, cheap) and summarising and tidying happen in the background.

The measurement: memory is the weakest agent capability

There is an unusually good independent source here. A university research group's function-calling leaderboard measures memory in a separate category, and measures three memory strategies on the same model:

• key-value storage — the agent writes into labelled slots and reads from them,
• vector retrieval — the stored chunks are searchable by meaning (the embeddings and vector search post covers this one),
• recursive summarisation — the agent periodically rewrites its own memory, more compactly.

The result (as of April 2026, 109 models):

(Own calculation from the public leaderboard's data table. The leaderboard is updated regularly.)

Three things can be read out of this, and all three come from measurement:

1. Memory is an order of magnitude weaker than the other agent capabilities. The same models that reach 86–91% on a single function call finish memory tasks at 29–74%. The average of the whole field does not reach 30% on any of the strategies.

2. On average, recursive summarisation wins — 28.2% against vector retrieval's 18.6% and key-value's 13.9%. At first glance that is surprising: summarisation loses information and still does better. The explanation is that raw storage preserves not only the information but the noise as well, and the agent has to work out for itself what to retrieve.

3. But there is no universally good architecture. Look at the fourth row of the table: the same model, in two different operating modes. In function-calling mode the vector strategy wins for it (63.23%), and by twelve points ahead of summarisation — while for most of the field it is the other way around. The memory architecture and the model are not independent choices. If you switch models, your memory strategy has to be measured again.

A second, academic measurement backs this up: on a 500-question benchmark designed specifically for long-term memory, commercial assistants and long-context models showed a 30% drop in accuracy when they had to remember something across several sessions.

The vendor numbers — and what they are worth

Two sets of figures circulate about this; both are worth knowing, and both are worth keeping in their place.

According to one model provider, the memory tool and context editing together gave a 39% improvement over the baseline; context editing on its own 29%. And on a 100-turn search task it made workflows completable that would otherwise fail as the context ran out — with 84% fewer tokens. (Internal, unpublished evaluation, with no sample size or methodology.)

According to a paper from a team building a memory product, their own solution gives a 26% relative improvement on a metric scored by a judge model, with 91% lower latency and more than 90% token savings compared with passing the full context. (The authors work for the company that builds the product being measured. The improvement is scored by a judge model, not by objective correctness.)

Behind the 84% and the 90% sits the same truth, and it is worth taking away even if you do not believe the exact numbers: memory is primarily a cost tool. Remembering does not make the agent smarter — it makes it cheaper, and able to run longer, because not everything has to be put in front of it again in every round.

Memory as an attack surface

This is the section most introductions leave out, even though the risk is highest here. Memory by definition carries trust from one run into the next — meaning a single successful attack achieves a lasting effect.

The most important measurement on this subject is what it is precisely because the attacker has no access to the memory. They can only ask questions and look at the answers. A March 2025 paper examined exactly that: it links the victim's later question to a harmful chain of thought through bridging steps, then gradually removes the explicit instructions until only the innocent-looking memory remains.

The cost of the attack: 10–15 queries per victim-target pair. Not a zero-day, not a leaked key — ordinary usage.

Another study, from 2024, measured backdoors hidden in memory and in the knowledge base across three real agent types: attack success rate above 80%, with a poisoning rate below 0.1% and less than 1% effect on normal operation. That last part is the crux: an agent with a poisoned memory behaves well until the trigger condition arrives.

Three documented cases showing the same pattern in the wild:

February 2025. A researcher showed that while summarising an untrusted document, the assistant can plant a conditional instruction: if the user later says a particular word, save something into long-term memory. The model then believes it is fulfilling a direct request from the user. In the proof-of-concept a nickname, a false age and an obviously untrue worldview got in. The vendor's classification: "low-likelihood, low-impact abuse risk." The researcher did not see it that way.

May 2026. A security team described a case where an everyday developer workflow — cloning a repository, approving the installation of a package — reached persistent memory and global settings, and influenced the highest-trust instruction layer as well. Their summary describes precisely why this is different from a plain injection: one incidental action shaped the model's future behaviour across sessions, projects and even restarts. The vendor's fix is instructive: they took user memories out of the system prompt, that is, out of the high-trust layer.

July 2026. The most recent case matters because there memory itself is the leak channel. The system builds a summary of a few paragraphs about the user from earlier conversations and inserts it into every new conversation. The attacker ran a website offering links organised alphabetically — and the model, navigating letter by letter, spelled the data out. The full name, the employer and the town where the user grew up all leaked. The user had never said that last one: the system inferred it from a mention of a local event, and what went into the memory was already the inference.

That final detail is the one worth remembering: what is in the memory is not what you said, but what the model inferred from it. And in that case the home town was also the answer to a security question.

The defence here is not inside the memory but on the way out — that is what the least-privilege agent post works through, and for the input side the prompt injection basics piece gives you the overview.

Writing to memory is data processing

Many people find this surprising, even though it follows directly from the text of the law: the EU data protection regulation explicitly names storage within the definition of processing. So if the agent writes personal data into its memory, that in itself is processing — with a legal basis, a purpose, a retention period and documentation.

There are four principles that translate directly into design decisions:

To that comes accountability: compliance also has to be demonstrable. So there should be a written retention policy for the agent's memory — even if that memory is nothing but a JSON file.

And there is one point that puts a direct requirement on the architecture. The right to erasure does not stop at a single record: the regulation explicitly extends it to copies and replications of the personal data. For an agent that means that when an erasure request arrives, you have to know which summary, which vector index and which earlier run's context the data ended up in.

That is technically hard — and precisely why it is worth writing memory with provenance: from which conversation, when, on what legal basis. You cannot reconstruct that afterwards. Where the data physically comes to rest is covered by the EU data residency post, in more depth.

(The EU data protection board issued an opinion in December 2024 on the data protection aspects of AI models. I found no regulatory guidance specifically on agent memory — legally this area is still open.)

What should we forget?

Five independent grounds, each with a concrete mechanism:

1. BY TIME
   Delete the memory files nobody has READ for a long
   time. Careful: by access, not by write - an old but
   constantly used memory is a good one.

2. BY SIZE
   Put an upper limit on file size, and on how many
   characters a single read returns. Without a limit
   the memory eats up the context window.

3. BY CONTENT
   Filter sensitive data out BEFORE the write, in code.
   The model "generally refuses" - that is no guarantee.

4. BY LEGAL BASIS
   When the purpose ends, so does the data. Inaccurate
   data has to be corrected or erased without delay.

5. BY TRUST LEVEL
   Do not push memory content up to the system prompt
   level. There the model treats it as an instruction.

The fifth point is the least obvious and the most expensive mistake. If the memory is part of the system prompt, then with one successful injection the attacker has acquired a persistent instruction, not just persistent data. The fix mentioned above reversed exactly that.

The how-to

1. WRITE DOWN WHAT YOU STORE - BEFORE YOU STORE IT
   Field by field: what, why, for how long, from where.
   Without this, your memory cannot be governed.

2. PICK A STRATEGY, BUT MEASURE IT TOO
   Recursive summarisation as the default.
   But a model switch can reshuffle the order.

3. STORE THE FACT AND THE INFERENCE SEPARATELY
   "He said this" and "from that I concluded" are
   not the same reliability, and they are not the
   same legal situation either.

4. WRITE NARROW, READ WIDE
   Save few things, but make sure that what you
   save can be found. The reverse fills the
   context up with noise.

5. AN EXPIRY ON EVERY FIELD
   Without a default lifetime the memory grows
   monotonically, and old errors never wear off.

6. HAVE A "WHAT DO YOU KNOW ABOUT ME" BUTTON
   Let the user see it and delete it. This is both
   a legal obligation and the best debugging there is.

7. REVIEW THE MEMORY LIKE A LOG
   Once a month, read through what has piled up.
   A poisoned memory behaves well - until it does not.

The sixth point is also the best investment commercially. When users are confronted with what the system stores about them, two kinds of reaction follow: either they correct it (and the agent gets better from then on), or they get alarmed (and then you were storing too much). Both are useful information.

Common misconceptions

• "A long context window replaces memory." It does not. The context window lasts within one run, memory lasts between runs. And long context degrades on its own too — the subagents and parallel work post brings the measurements for that.
• "Memory makes the agent smarter." Mainly it makes it cheaper and able to run longer. The strongest numbers for accuracy improvement come from vendors or interested parties.
• "Let's save everything, filtering can wait." The measurement says raw storage is the weakest strategy, and the law says you cannot store unnecessary data. This is the rare case where the two considerations point the same way.
• "The model will not write out sensitive data anyway." The documentation's careful wording is that it "generally refuses". For a strong guarantee you have to filter in code, before the write.
• "Only someone with access can attack the memory." The measurement shows a 98.2% injection success rate achievable purely through queries, from fifteen questions.
• "If I delete the record, it is deleted." The record also made it into summaries, vector indexes and the traces of earlier runs. To erase it you have to know where it went.

What to take away

Memory is the least solved part of agents. The numbers say so unambiguously: the field average does not reach 30% on any strategy, and a different architecture wins for each model — meaning there is no proven recipe here, only measurement.

The practical consequence is therefore not "do not use memory", but treat memory like a database, not like a convenience feature. It has a schema, a retention period, access control, and from time to time you have to look through what has gone into it.

And the cheapest check you can run today: read what the assistant you use every day stores about you. There is a good chance it contains an inference you never spoke out loud — and a good chance it contains something that is no longer true.

Why reliability degrades with step count is what the agent reliability math post sets out; assembling the context deliberately is the theme of the context engineering piece.