Context is not memory: why the distinction matters

When an agent recalls a customer preference from last month, engineers often say that it “used context.” When a model receives a 100-page policy manual in the current prompt, they may say it “has memory.” The terms sound interchangeable, but the systems behave differently.

Context is what the model can use during a particular inference. Memory is the governed lifecycle that decides what persists across inferences and how it returns. A long prompt can disappear after one call. A six-word memory can influence decisions for years. Architecture, privacy and evaluation choices depend on knowing which one you are building.

Draw the boundary at persistence

A context window is a temporary workspace. It may contain the current instruction, recent conversation, retrieved documents, tool results, a plan and system rules. The model processes that input and produces an output; the application then decides what, if anything, survives.

Memory begins when the system deliberately carries information across that boundary. It needs at least three operations:

1. Write: select and encode something for later use.
2. Manage: retain, consolidate, correct, scope or delete it.
3. Read: find and place a useful memory into a future context.

The ACM survey of LLM-agent memory examines memory through its sources, forms and operations, and emphasizes that agent memory needs design and evaluation beyond simply extending an input. This distinction gives teams a concrete review target: the context assembly path and the memory lifecycle are connected, but neither substitutes for the other.

Four stores are commonly collapsed into one

The word “memory” often hides different records with different owners.

Information	Example	Authoritative owner	Appropriate lifecycle
Working context	Tool response used in the current step	Agent runtime	Discard or retain with the execution trace
Workflow state	Payment approval is pending	Transaction or workflow system	Persist until a defined terminal state and audit period
User preference	Customer prefers email updates	Customer/profile system	Purpose-limited, correctable and revocable
Learned episode	A previous plan failed after a timeout	Agent memory service	Curated, versioned and tested before reuse

Putting all four into a vector database does not make them equivalent. A similarity search may be suitable for finding an old episode, but not for deciding whether a payment is currently approved. A conversation summary may help continuity, but it should not overwrite the customer profile. A transaction state should be retrieved by a stable identifier, not by semantic resemblance.

The architecture should preserve those ownership boundaries even if one storage technology supports several of them.

More context does not guarantee usable recall

Larger context windows can reduce the need to fetch or summarise, but capacity is not the same as reliable use. The peer-reviewed “Lost in the Middle” study found that model performance could change materially with the position of relevant information and was often weaker when that information appeared in the middle of long input. It also found that adding more retrieved documents could reach diminishing returns before retriever recall did.

That observation changes evaluation. A team should not treat “fits inside the window” as an acceptance criterion. It should vary:

• where the relevant fact appears;
• how much irrelevant or conflicting material surrounds it;
• whether the fact is repeated with different timestamps;
• whether the model should use, ignore or ask to verify it; and
• whether the same assembly policy works after a model change.

The right question is not how many tokens the system can send. It is whether the right evidence predictably influences the right decision.

Memory writes deserve the same scrutiny as memory reads

Many memory evaluations ask, “Did the agent retrieve the fact?” A more consequential failure may happen earlier: the system stored the wrong fact, stripped its source, merged two people, or promoted an unverified inference into a durable profile.

A memory write policy should record:

• the originating person, system or document;
• the exact observation and any interpretation made from it;
• the purpose and permitted future uses;
• the subject or workflow scope;
• confidence and verification status where relevant;
• creation, review and expiry times;
• the component and version that transformed it; and
• links to correction or deletion actions.

Do not let a model’s polished summary erase disagreement. If one message says a customer prefers email and a later message requests phone contact for a specific incident, the memory layer needs scope and time, not a single “latest truth” synthesized without provenance.

Retrieval is an authorisation decision

Memory access is often implemented as a relevance problem: embed the query, return the nearest records. In a real organisation, it is also an authorisation and purpose problem. A record can be semantically relevant and still be unavailable to this user, agent, task or geography.

Apply access controls before retrieved text enters the model context. Log the requesting principal, declared purpose, policy decision and records returned. Where sensitive data is involved, prefer references or minimised fields over copying whole records into an agent-owned store.

The NIST Privacy Framework frames privacy risk as arising from data processing that can affect people. For agent memory, that means retention and retrieval are not merely optimisation choices. They determine how information about people is reused, combined and exposed.

Design correction and forgetting at the beginning

A memory system that can only add is unfinished. It needs behaviour for correction, expiry, deletion and derived records. If a source record is corrected, identify which summaries or embeddings were derived from it. If a retention period ends, remove or make inaccessible all active representations required by the policy—not only the visible text row.

For deletion-sensitive designs, maintain a lineage map from source to chunks, embeddings, summaries and cached prompts. Test that expired information does not return through semantic retrieval, backup restoration or a regenerated summary. The goal is not a universal claim that a model has “forgotten”; it is evidence that the application’s controlled stores and retrieval paths honor the defined lifecycle.

The NIST Generative AI Profile recommends documenting data provenance, monitoring risks and aligning controls to the deployment context. Those practices matter on both sides of memory: what is written and what is later exposed.

Evaluate a memory loop, not a recall quiz

A useful test suite follows an item through its lifecycle:

Stage	Test question	Example failure
Write	Should this observation become memory?	Unverified speculation is stored as fact
Scope	Who and what task may use it?	One account’s preference appears in another account
Consolidate	How are conflicts and repetition handled?	New evidence silently erases a valid exception
Retrieve	Does the right record return under realistic noise?	An older, more similar record outranks the current one
Apply	Does the agent use it appropriately?	A preference is treated as permission
Correct	Do dependent representations change?	The text is fixed but its old embedding remains active
Expire	Does it stop influencing behaviour?	A restored cache reintroduces an expired record

Run the suite across sessions and model versions, not only within one conversation. Include negative cases where the correct behaviour is not to remember or retrieve.

Choose a memory promise the system can keep

Before release, write a plain-language memory contract: what the agent retains, why, for how long, who can use it, how a person corrects it and which records remain authoritative elsewhere. Then connect each promise to an enforcement point and a test.

If the team cannot explain a memory’s provenance or delete it from active use, narrow the memory feature. Session-only context, explicit saved notes or references to an existing system of record may be the better product. Persistent personalization is valuable only when its lifecycle is more dependable than the convenience it adds.

Context helps a model act now. Memory influences what an agent may do later. Treating the difference as architecture—not terminology—makes both easier to govern.