Memory deletion and correction testing
Determine whether the requested change propagates through the memory-enabled workflow
When a user or data owner requests deletion or correction, this area becomes relevant. For the privacy engineering and deployment owner, the concern is specific: a corrected fact remains in a summary and is written back into memory. The deployment evaluation starts from that consequence and follows the relevant evidence through the customer workflow. Test the action and its aftermath. Verification should inspect configured stores, derived artefacts, and later behaviour within the authorised boundary, while recording any provider or backup pathway that could not be examined.
Connect the operating question to inspectable evidence
- Privacy engineering and deployment owner
Owns the bounded decision and the consequence of acting.
- Deletion and correction requests
Defines the starting context for the proposed work.
- Primary store update
Exposes the first material uncertainty to examine.
- Unresolved lifecycle findings
Carries unresolved conditions into an accountable next action.
Decision at stake
Determine whether the requested change propagates through the memory-enabled workflow
Make the decision explicit
- Privacy engineering and deployment owner
- A user or data owner requests deletion or correction.
- Decision
- Whether the requested change propagates through the memory-enabled workflow.
- Primary store update
- Use deletion and correction requests to establish the relevant starting condition.
For the privacy engineering and deployment owner, this becomes relevant when a user or data owner requests deletion or correction. The immediate decision is whether the requested change propagates through the memory-enabled workflow. The scope should be narrow enough that the evaluation team can identify a meaningful alternative, including continuing the current approach if the available evidence already supports it.
The practical failure to challenge is that a corrected fact remains in a summary and is written back into memory. That concern gives the work a concrete direction: establish what the relevant boundary should do, inspect how it behaves or would operate, and identify the owner of any gap. It should not be diluted into a broad transformation exercise before the specific question has been answered.
Coverage
Examine primary store update alongside derived copy propagation
Keep the review distinctions visible
- Primary store update
- Verify the authoritative memory record after an authorised request.
- Derived copy propagation
- Follow the change into indexes, summaries and other scoped copies.
- Later retrieval and regeneration
- Retry retrieval and replay to check whether old information returns.
Primary store update
For this scope, primary store update should be examined alongside deletion and correction requests. Establish the original entry and expected correction or deletion effect before running the request. Verify the primary store using authorised inspection, then distinguish that storage observation from later agent behaviour. A successful request acknowledgement alone does not establish that the intended state change occurred.
Derived copy propagation
For this scope, derived copy propagation should be examined alongside derived-store inventory. Inventory derived copies relevant to the deployment, including summaries, indexes, and caches. Test whether the requested change reaches them and whether their refresh rules are visible. The proposed scope should say which paths were observed and which require evidence from another provider or system owner.
Later retrieval and regeneration
For this scope, later retrieval and regeneration should be examined alongside verification and replay cases. Repeat tasks that previously used the old information and inspect any new writes. A deleted assertion can be regenerated from a remaining representation. The test should therefore connect storage changes to subsequent retrieval and use, while preserving uncertainty about any lifecycle path outside the authorised boundary.
Prerequisites
Start with deletion and correction requests
Give each input a specific job
- Deletion and correction requests
- Defines the operating boundary.
- Derived-store inventory
- Examine derived copy propagation.
- Verification and replay cases
- Shows conditions affecting later retrieval and regeneration.
- Product or platform owner
- Confirm access and a reviewer who can resolve gaps.
The starting material is deletion and correction requests, derived-store inventory, and verification and replay cases. Review these together rather than assuming one document tells the whole story. A stated rule may differ from a configured behaviour, and a proposed responsibility may not yet have an available owner. Identify those differences before selecting the detailed work.
Participation should include a product or platform decision owner, a technical lead, a data owner, and a competent reviewer for the relevant boundary. The customer should define which information and environments may be used. Synthetic or sanitised examples can support preparation when they preserve the relevant behaviour; if they omit a material dependency, record the limitation and agree how the missing evidence would be obtained.
How the work proceeds
Tie the deployment finding to observable behaviour
Follow the evidence through the work
- Primary store update
Use deletion and correction requests to define the decision criteria.
- Derived copy propagation
Inspect derived-store inventory and retain observations that challenge the expected result.
- Later retrieval and regeneration
Use verification and replay cases to identify unresolved conditions and a responsible owner.
- Unresolved lifecycle findings
Link the recommendation to its evidence, remaining limits and next validation condition.
Define a verification sequence before executing the request: establish the original fact, apply correction or deletion, inspect in-scope representations, and repeat relevant tasks. Include delayed and derived pathways where authorised. The proposed test should record which outcomes demonstrate data change and which demonstrate changed agent behaviour.
A practical first slice would focus on primary store update and then follow its consequences into derived copy propagation. Keep later retrieval and regeneration visible as a separate review question rather than assuming it is satisfied by the first result. The evaluation team should record observations, competing explanations, and changes to the original assumptions. A second reviewer can challenge whether the evidence supports the recommendation and whether an apparently successful example omitted the condition that matters most.
Illustrative scenario
Authorised deletion: examine what changes the decision
Examine the constructed decision boundary
Illustrative scenario
- Authorised deletion
A memory record is removed while derived copies may remain in retrieval paths.
- Propagation check
Inspect summaries and indexes, then retry retrieval and regeneration.
- Unresolved state
Keep the incomplete copy transition visible in the lifecycle finding.
Consider an illustrative situation in which a synthetic customer corrects an address and requests removal of an obsolete note before the agent resumes work. This is a constructed example, not a client engagement or an observed Softlander outcome.
For the address correction, an illustrative run would check that the new address governs later authorised work and that the obsolete note does not reappear through a summary. Any unexamined backup or provider retention path remains outside the observation. Qualified privacy interpretation is separate from the technical findings.
The useful learning would be the difference between the expected behaviour and the behaviour or responsibility actually demonstrated. If the exercise leaves later retrieval and regeneration unresolved, that should remain a named next step. A recommendation to narrow the scope, improve a prerequisite, or hold the decision can be appropriate when the example exposes a material dependency.
Indicative artefacts
Use unresolved lifecycle findings to make the next action explicit
Every artefact needs an operating use
Illustrative working format
- Deletion and correction test pack
- Boundary — deletion and correction requests.
- Propagation evidence
- Findings about derived copy propagation.
- Unresolved lifecycle findings
- Action on later retrieval and regeneration.
Indicative outputs include deletion and correction test pack, propagation evidence, and unresolved lifecycle findings. Their form should follow the decision and the customer’s existing operating records. The first should establish the assessed boundary and relevant observations; the second should make material gaps or dependencies inspectable; the third should identify the action, owner, and evidence needed to revisit the decision.
Acceptance should be tested through use: can the responsible owner explain what was examined, identify an unresolved issue, and act on the next step without reconstructing the whole engagement? Findings should preserve counterevidence and limits. A polished document is insufficient if it does not help resolve the customer’s specific question.
Scope and authority
A deployment finding remains bounded to the tested conditions
Keep analytical work separate from decision authority
- Authorised deployment cases and traces
- Can include deletion and correction test pack.
- Product or platform owner
- The customer retains release authority and owns remediation priorities.
- Universal vendor or component assurance
- Cannot be inferred from propagation evidence.
The proposed evaluation concerns the named deployment, configuration, cases, and authorised environment. Findings do not certify a component vendor, establish universal safety, or approve release. Unexamined provider paths and specialist questions remain explicit limitations.
Ways to engage
Related engagement scopes
AI Agent Evaluation Evidence Diagnostic
Find the gaps between your release decision and the evidence behind it. Start with one agent workflow and a bounded set of questions.
AI Agent Evaluation Foundation
Turn scattered tests into a repeatable release discipline. Connect the behaviours that matter to datasets, evaluation criteria and evidence your team can maintain.
AI Agent Component Evaluation
Find where agent behaviour breaks down. Follow memory, retrieval and delegation through your workflow, then isolate the components that need attention.
Continue exploring
Related insights and resources
Engineer the harness, not just the prompt
Define the contracts around an agent's decisions: what state it trusts, which actions can execute, how interrupted work resumes, and what evidence an operator needs to recover safely.
What would justify releasing this agent?
Replace a single pass rate with a reviewable chain of requirements, cases, observations and judgements. Distinguish task quality, control failures and uncertainty before increasing an agent's exposure.
The evidence behind the go-live decision
A working paper for leaders who need to explain why an agent workflow is ready for its next release. Includes a proposed evidence-pack structure, decision table and review agenda.