Memory poisoning assessment
Determine whether malicious or misleading writes can influence later authorised tasks
When untrusted information may enter persistent agent memory, this area becomes relevant. For the AI security and memory product owner, the concern is specific: a malicious statement is stored as trusted memory and affects future tool decisions. The deployment evaluation starts from that consequence and follows the relevant evidence through the customer workflow. Follow the attack across interactions instead of testing only the immediate response. The proposed assessment should separate an attempted write, a persisted entry, a retrieved entry, and a consequential downstream effect.
Connect the operating question to inspectable evidence
- AI security and memory product owner
Owns the bounded decision and the consequence of acting.
- Memory write eligibility rules
Defines the starting context for the proposed work.
- Write acceptance
Exposes the first material uncertainty to examine.
- Control gap recommendations
Carries unresolved conditions into an accountable next action.
Decision at stake
Determine whether malicious or misleading writes can influence later authorised tasks
Make the decision explicit
- AI security and memory product owner
- Untrusted information may enter persistent agent memory.
- Decision
- Whether malicious or misleading writes can influence later authorised tasks.
- Write acceptance
- Use memory write eligibility rules to establish the relevant starting condition.
For the AI security and memory product owner, this becomes relevant when untrusted information may enter persistent agent memory. The immediate decision is whether malicious or misleading writes can influence later authorised tasks. The scope should be narrow enough that the evaluation team can identify a meaningful alternative, including continuing the current approach if the available evidence already supports it.
The practical failure to challenge is that a malicious statement is stored as trusted memory and affects future tool decisions. That concern gives the work a concrete direction: establish what the relevant boundary should do, inspect how it behaves or would operate, and identify the owner of any gap. It should not be diluted into a broad transformation exercise before the specific question has been answered.
Coverage
Examine write acceptance alongside provenance and trust handling
Keep the review distinctions visible
- Write acceptance
- Observe whether untrusted content becomes a stored memory item.
- Provenance and trust handling
- Preserve source trust through summarization and later retrieval.
- Persistence of hostile influence
- Follow an accepted hostile memory into a later task's action path.
Write acceptance
For this scope, write acceptance should be examined alongside memory write eligibility rules. Challenge write eligibility with untrusted content that asks to be remembered as a future instruction. Record whether the system rejects, qualifies, or stores it. The proposed assessment should distinguish a permitted factual memory from a write that elevates untrusted text into continuing authority.
Provenance and trust handling
For this scope, provenance and trust handling should be examined alongside untrusted input examples. Inspect the trust and provenance attached to a stored entry. A label matters only if later retrieval and use respect its meaning. Test whether an instruction originating in an external document is treated differently from an authorised user preference when a subsequent task is performed.
Persistence of hostile influence
For this scope, persistence of hostile influence should be examined alongside subsequent task traces. Repeat a legitimate task after the poisoning attempt and observe any changed reasoning or tool behaviour. Immediate refusal does not answer the persistence question. Include an authorised correction or removal step so the owner can see whether harmful influence can be located and contained within the tested lifecycle.
Prerequisites
Start with memory write eligibility rules
Give each input a specific job
- Memory write eligibility rules
- Defines the operating boundary.
- Untrusted input examples
- Examine provenance and trust handling.
- Subsequent task traces
- Shows conditions affecting persistence of hostile influence.
- Product or platform owner
- Confirm access and a reviewer who can resolve gaps.
The starting material is memory write eligibility rules, untrusted input examples, and subsequent task traces. Review these together rather than assuming one document tells the whole story. A stated rule may differ from a configured behaviour, and a proposed responsibility may not yet have an available owner. Identify those differences before selecting the detailed work.
Participation should include a product or platform decision owner, a technical lead, a data owner, and a competent reviewer for the relevant boundary. The customer should define which information and environments may be used. Synthetic or sanitised examples can support preparation when they preserve the relevant behaviour; if they omit a material dependency, record the limitation and agree how the missing evidence would be obtained.
How the work proceeds
Tie the deployment finding to observable behaviour
Follow the evidence through the work
- Write acceptance
Use memory write eligibility rules to define the decision criteria.
- Provenance and trust handling
Inspect untrusted input examples and retain observations that challenge the expected result.
- Persistence of hostile influence
Use subsequent task traces to identify unresolved conditions and a responsible owner.
- Control gap recommendations
Link the recommendation to its evidence, remaining limits and next validation condition.
Create a controlled sequence in which untrusted input attempts to become a future instruction. Inspect whether the write is accepted, how its trust and provenance are recorded, and which later task retrieves it. The proposed assessment should also test the correction path after a poisoned entry is identified.
A practical first slice would focus on write acceptance and then follow its consequences into provenance and trust handling. Keep persistence of hostile influence visible as a separate review question rather than assuming it is satisfied by the first result. The evaluation team should record observations, competing explanations, and changes to the original assumptions. A second reviewer can challenge whether the evidence supports the recommendation and whether an apparently successful example omitted the condition that matters most.
Illustrative scenario
False account instruction: control-specific response
Examine the constructed decision boundary
Illustrative scenario
- False account instruction
An untrusted message asks the agent to retain an instruction for later.
- Delayed legitimate request
Separate the write attempt from a later task and trace any persisted influence.
- Control-specific response
Target write eligibility, retrieval or action enforcement according to the observed effect.
Consider an illustrative situation in which an agent reads an untrusted message that asks it to remember a false account instruction for later use. This is a constructed example, not a client engagement or an observed Softlander outcome.
For the false-account-instruction example, the illustrative test would separate the initial interaction from a later legitimate request. This exposes persistence that an immediate refusal test could miss. The owner would receive evidence about the attempted influence and any observed effect, with recommendations tied to write eligibility, retrieval, or action enforcement.
The useful learning would be the difference between the expected behaviour and the behaviour or responsibility actually demonstrated. If the exercise leaves persistence of hostile influence unresolved, that should remain a named next step. A recommendation to narrow the scope, improve a prerequisite, or hold the decision can be appropriate when the example exposes a material dependency.
Indicative artefacts
Use control gap recommendations to make the next action explicit
Every artefact needs an operating use
Illustrative working format
- Memory poisoning case set
- Boundary — memory write eligibility rules.
- Write-to-effect traces
- Findings about provenance and trust handling.
- Control gap recommendations
- Action on persistence of hostile influence.
Indicative outputs include memory poisoning case set, write-to-effect traces, and control gap recommendations. Their form should follow the decision and the customer’s existing operating records. The first should establish the assessed boundary and relevant observations; the second should make material gaps or dependencies inspectable; the third should identify the action, owner, and evidence needed to revisit the decision.
Acceptance should be tested through use: can the responsible owner explain what was examined, identify an unresolved issue, and act on the next step without reconstructing the whole engagement? Findings should preserve counterevidence and limits. A polished document is insufficient if it does not help resolve the customer’s specific question.
Scope and authority
A deployment finding remains bounded to the tested conditions
Keep analytical work separate from decision authority
- Authorised deployment cases and traces
- Can include memory poisoning case set.
- Product or platform owner
- The customer retains release authority and owns remediation priorities.
- Universal vendor or component assurance
- Cannot be inferred from write-to-effect traces.
The proposed evaluation concerns the named deployment, configuration, cases, and authorised environment. Findings do not certify a component vendor, establish universal safety, or approve release. Unexamined provider paths and specialist questions remain explicit limitations.
Ways to engage
Related engagement scopes
AI Agent Evaluation Evidence Diagnostic
Find the gaps between your release decision and the evidence behind it. Start with one agent workflow and a bounded set of questions.
AI Agent Evaluation Foundation
Turn scattered tests into a repeatable release discipline. Connect the behaviours that matter to datasets, evaluation criteria and evidence your team can maintain.
AI Agent Component Evaluation
Find where agent behaviour breaks down. Follow memory, retrieval and delegation through your workflow, then isolate the components that need attention.
Continue exploring
Related insights and resources
Engineer the harness, not just the prompt
Define the contracts around an agent's decisions: what state it trusts, which actions can execute, how interrupted work resumes, and what evidence an operator needs to recover safely.
What would justify releasing this agent?
Replace a single pass rate with a reviewable chain of requirements, cases, observations and judgements. Distinguish task quality, control failures and uncertainty before increasing an agent's exposure.
The evidence behind the go-live decision
A working paper for leaders who need to explain why an agent workflow is ready for its next release. Includes a proposed evidence-pack structure, decision table and review agenda.