Can your agent truly forget?

An agent can appear to forget while its data remains everywhere that matters. The chat disappears, but a memory service retains the preference. A vector index still retrieves the document. An observability platform holds the prompt. An evaluation set contains the conversation. A provider backup will age out later. A fine-tuned model may have learned a pattern that cannot be removed with a database command.

“Delete” is therefore not one operation. It is a promise about a data lineage. Before making that promise, an organisation needs to know where information travelled, what each layer can remove, how long propagation takes and what evidence demonstrates the result.

Define what forgetting must mean

Different user expectations and obligations may require different outcomes:

• Hide: content is no longer visible in the interface.
• Deactivate: the agent stops using an item, but it remains retained.
• Delete from active stores: the record is removed from production systems.
• Delete derived references: embeddings, summaries, features and evaluation copies are removed or regenerated.
• Expire from backups: retained copies become inaccessible for normal use and are deleted through the backup lifecycle.
• Change model behaviour: a trained model no longer reveals or relies on the information to a defined test threshold.

Do not use these terms interchangeably. A product can offer immediate interface deletion while protected backups expire later, but that timing and restriction should be accurately documented. A memory toggle can prevent future writes without deleting earlier data.

Whether a legal right to erasure applies and what exceptions or response periods are available depends on jurisdiction, role, purpose and facts. Obtain qualified advice for specific requests. The engineering task is to make the real behaviour inspectable so legal and product teams do not reason from an inaccurate diagram.

Map every persistence layer

Trace data from entry to all direct and derived stores:

Layer	Examples	Deletion question
Interaction	Chat, attachments, voice transcript	Is the authoritative record deleted or only hidden?
Working state	Task scratchpad, queue, session cache	When does it expire, including failed tasks?
Durable memory	Preferences, summaries, facts	Can items be located by subject and provenance?
Retrieval	Documents, chunks, embeddings, caches	Does source deletion remove every searchable derivative?
Tools	CRM, ticket, calendar, file store	Which system is authoritative and who propagates deletion?
Observability	Prompts, traces, errors, analytics	Is content redacted, sampled and retention-limited?
Evaluation	Regression cases, human review samples	Do copied cases inherit deletion obligations?
Learning	Fine-tuning set, feedback store, model	Can the data and resulting influence be addressed?
Resilience	Replicas, backups, archives	When do copies age out and who can restore them?
Suppliers	Model, search and monitoring services	What does each provider retain and how is deletion verified?

Include identifiers needed to find the records. If minimisation removed direct identity, the organisation may genuinely be unable to locate a person’s data without additional information. That trade-off should be designed and explained rather than discovered during a request.

Keep provenance with memory and derivatives

Deletion is much harder when derived records have lost their source relationship. Store lineage from a memory summary or embedding back to its source record, tenant, purpose and retention class. When a source changes, use that lineage to invalidate, delete or rebuild derivatives.

Avoid one-way copying into evaluation sets. A test case taken from production should carry a protected provenance key and deletion policy. If it is transformed into a genuinely anonymous record, document the method and assessment; pseudonymisation alone does not necessarily put data outside data-protection scope.

For agent memory, distinguish facts explicitly supplied by a user, inferences produced by the system and authoritative business records. Deleting one conversation should not silently delete a transaction record that must be retained, but neither should a derived “preference” survive because nobody recorded where it came from.

Design a deletion state machine

A deletion workflow should have explicit states rather than a fire-and-forget API call:

1. Validated: identity, authority and scope are confirmed.
2. Located: direct records and derivatives are enumerated.
3. Restricted: data is blocked from normal use while processing where appropriate.
4. Deleted or transformed: each active store reports its defined result.
5. Propagated: suppliers and downstream systems acknowledge action.
6. Verified: retrieval, memory and output tests no longer expose the targeted data within the stated scope.
7. Backups pending: protected copies remain inaccessible until scheduled expiry, if applicable.
8. Closed: the organisation records a minimal audit outcome and informs the requester as required.

Handle partial failure. If a vector index delete succeeds but the memory store times out, the request should remain open and the data restricted rather than displaying a misleading success state.

Treat learned models as a different problem

Deleting a row from a training dataset does not automatically reverse its influence on a trained model. Options can include retraining without the data, replacing a model at the next training cycle, machine-unlearning techniques or effective safeguards around the model. Each option needs evidence for the system and risk in question.

CNIL’s recommendations on exercising rights for AI discuss retraining and possible alternative measures where retraining is disproportionate, together with the need to assess effectiveness and inform individuals. Its broader AI development guidance stresses that the proportionality of measures depends on sensitivity, extraction risk and impact.

Do not reduce this to “models cannot forget” or “unlearning solves it.” Some systems can be retrained; some data may not be identifiable in the model; some safeguards may reduce disclosure without removing influence. Record the model, data path, threat, test and residual uncertainty.

The EDPB’s Opinion 28/2024 likewise treats anonymity of AI models as a case-by-case assessment and considers whether personal data may be extracted or obtained from queries. Teams should not infer anonymity from difficulty alone.

Test forgetting as an end-to-end property

Use synthetic, uniquely traceable records to test the process without exposing real people. Seed the record through the same interfaces, allow expected derivatives to form, invoke deletion and inspect each mapped layer.

Verify:

• the user interface and authoritative stores;
• memory search by identifier and semantic query;
• vector and keyword retrieval, including caches;
• tool systems and downstream events;
• logs and observability under normal access;
• evaluation and feedback datasets;
• supplier acknowledgements;
• backup restriction and scheduled expiry;
• model outputs under reasonable extraction attempts where relevant.

Record timing and failure behaviour. A test that works in an empty development database may fail against production-scale indexes or delayed event queues. Re-run after storage, provider, model or memory changes.

Avoid deletion that destroys accountability

Erasure and auditability can conflict if handled crudely. Deleting every trace of a request can make it impossible to prove completion or investigate abuse. Preserve a minimal record of request ID, validated authority, scope, timestamps, systems addressed and outcome without retaining the content that was meant to be removed.

Separate operational logs from compliance evidence and restrict both. The NIST Privacy Framework helps organisations treat privacy effects as enterprise risk and coordinate Identify, Govern, Control, Communicate and Protect activities; use that structure to balance individual impact, security and accountability rather than assuming one control settles all three.

Make the product promise match the architecture

Expose clear choices: forget this memory, delete this conversation, clear all retained preferences or close the account. Explain material delays and exclusions in plain language. Do not say “deleted forever” if protected backups persist for a defined period or a supplier cannot support that claim.

Minimise collection and retention so deletion remains tractable. CNIL’s data-management guidance connects relevance, storage duration, maintenance and lifecycle controls. The cheapest deletion request is often the data the agent never copied into another layer.

An agent can credibly forget only within a defined technical and operational boundary. Map that boundary, make every layer report its state, test propagation and describe residual copies honestly. The resulting answer may be more nuanced than a delete button—but it will be true.