Context poisoning and injection assessment
Determine whether hostile context can redirect behaviour beyond its permitted influence
When untrusted documents or tool outputs enter the agent's context, this area becomes relevant. For the AI security and retrieval owner, the concern is specific: retrieved content is treated as authority to change the task or reveal restricted information. The deployment evaluation starts from that consequence and follows the relevant evidence through the customer workflow. Test the route from untrusted text to agent decision and tool effect. Keep the evaluation within approved environments and distinguish a model repeating hostile content from the system actually exceeding its authorised behaviour.
Connect the operating question to inspectable evidence
- AI security and retrieval owner
Owns the bounded decision and the consequence of acting.
- Trust boundary map
Defines the starting context for the proposed work.
- Instruction confusion
Exposes the first material uncertainty to examine.
- Mitigation verification backlog
Carries unresolved conditions into an accountable next action.
Decision at stake
Determine whether hostile context can redirect behaviour beyond its permitted influence
Make the decision explicit
- AI security and retrieval owner
- Untrusted documents or tool outputs enter the agent's context.
- Decision
- Whether hostile context can redirect behaviour beyond its permitted influence.
- Instruction confusion
- Use trust boundary map to establish the relevant starting condition.
For the AI security and retrieval owner, this becomes relevant when untrusted documents or tool outputs enter the agent’s context. The immediate decision is whether hostile context can redirect behaviour beyond its permitted influence. The scope should be narrow enough that the evaluation team can identify a meaningful alternative, including continuing the current approach if the available evidence already supports it.
The practical failure to challenge is that retrieved content is treated as authority to change the task or reveal restricted information. That concern gives the work a concrete direction: establish what the relevant boundary should do, inspect how it behaves or would operate, and identify the owner of any gap. It should not be diluted into a broad transformation exercise before the specific question has been answered.
Coverage
Examine instruction confusion alongside source trust handling
Keep the review distinctions visible
- Instruction confusion
- Keep external source text separate from trusted instructions.
- Source trust handling
- Change source trust while preserving the requested task.
- Unauthorised action attempts
- Inspect the actual attempted action as well as the model's refusal text.
Instruction confusion
For this scope, instruction confusion should be examined alongside trust boundary map. Test whether the agent treats retrieved instructions as information or as authority to alter the legitimate task. Preserve the original request and permitted actions throughout the trace. A finding should identify the transition where hostile text gained influence, rather than merely quote the adversarial content in the final answer.
Source trust handling
For this scope, source trust handling should be examined alongside authorised adversarial fixtures. Vary the apparent source and formatting of authorised hostile fixtures while keeping their intended manipulation explicit. Inspect how trust metadata and assembly rules affect the result. The proposed test should distinguish a control that prevents influence from one that merely reduces it in a particular example.
Unauthorised action attempts
For this scope, unauthorised action attempts should be examined alongside action and refusal expectations. Observe attempted and completed actions separately. A model may propose an unauthorised call that another component denies, or it may refuse before invocation. Both observations matter, but they support different conclusions about behaviour and enforcement within the tested deployment boundary.
Prerequisites
Start with trust boundary map
Give each input a specific job
- Trust boundary map
- Defines the operating boundary.
- Authorised adversarial fixtures
- Examine source trust handling.
- Action and refusal expectations
- Shows conditions affecting unauthorised action attempts.
- Product or platform owner
- Confirm access and a reviewer who can resolve gaps.
The starting material is trust boundary map, authorised adversarial fixtures, and action and refusal expectations. Review these together rather than assuming one document tells the whole story. A stated rule may differ from a configured behaviour, and a proposed responsibility may not yet have an available owner. Identify those differences before selecting the detailed work.
Participation should include a product or platform decision owner, a technical lead, a data owner, and a competent reviewer for the relevant boundary. The customer should define which information and environments may be used. Synthetic or sanitised examples can support preparation when they preserve the relevant behaviour; if they omit a material dependency, record the limitation and agree how the missing evidence would be obtained.
How the work proceeds
Tie the deployment finding to observable behaviour
Follow the evidence through the work
- Instruction confusion
Use trust boundary map to define the decision criteria.
- Source trust handling
Inspect authorised adversarial fixtures and retain observations that challenge the expected result.
- Unauthorised action attempts
Use action and refusal expectations to identify unresolved conditions and a responsible owner.
- Mitigation verification backlog
Link the recommendation to its evidence, remaining limits and next validation condition.
Construct authorised adversarial fixtures that target distinct boundaries: task redirection, instruction priority, information disclosure, and tool action. Preserve the original legitimate request and expected permitted behaviour. The proposed assessment should record attempted influence and observed system effect separately so severity reflects what actually happened.
A practical first slice would focus on instruction confusion and then follow its consequences into source trust handling. Keep unauthorised action attempts visible as a separate review question rather than assuming it is satisfied by the first result. The evaluation team should record observations, competing explanations, and changes to the original assumptions. A second reviewer can challenge whether the evidence supports the recommendation and whether an apparently successful example omitted the condition that matters most.
Illustrative scenario
Hostile knowledge article: boundary-specific finding
Examine the constructed decision boundary
Illustrative scenario
- Hostile knowledge article
A synthetic source contains instructions conflicting with the user's permitted task.
- Cross-layer trace
Follow retrieval, assembly, response and attempted tool action.
- Boundary-specific finding
Distinguish model refusal from tool denial and remediate the layer the evidence identifies.
Consider an illustrative situation in which a synthetic knowledge article contains instructions that conflict with the user’s authorised request. This is a constructed example, not a client engagement or an observed Softlander outcome.
For the hostile knowledge article, an illustrative trace would show retrieval, context assembly, agent response, and any attempted tool call. A refusal at one layer and a denial at another provide different evidence. Recommendations should identify the boundary needing improvement instead of implying that a single prompt change resolves every injection risk.
The useful learning would be the difference between the expected behaviour and the behaviour or responsibility actually demonstrated. If the exercise leaves unauthorised action attempts unresolved, that should remain a named next step. A recommendation to narrow the scope, improve a prerequisite, or hold the decision can be appropriate when the example exposes a material dependency.
Indicative artefacts
Use mitigation verification backlog to make the next action explicit
Every artefact needs an operating use
Illustrative working format
- Context injection cases
- Boundary — trust boundary map.
- Trust-boundary traces
- Findings about source trust handling.
- Mitigation verification backlog
- Action on unauthorised action attempts.
Indicative outputs include context injection cases, trust-boundary traces, and mitigation verification backlog. Their form should follow the decision and the customer’s existing operating records. The first should establish the assessed boundary and relevant observations; the second should make material gaps or dependencies inspectable; the third should identify the action, owner, and evidence needed to revisit the decision.
Acceptance should be tested through use: can the responsible owner explain what was examined, identify an unresolved issue, and act on the next step without reconstructing the whole engagement? Findings should preserve counterevidence and limits. A polished document is insufficient if it does not help resolve the customer’s specific question.
Scope and authority
A deployment finding remains bounded to the tested conditions
Keep analytical work separate from decision authority
- Authorised deployment cases and traces
- Can include context injection cases.
- Product or platform owner
- The customer retains release authority and owns remediation priorities.
- Universal vendor or component assurance
- Cannot be inferred from trust-boundary traces.
The proposed evaluation concerns the named deployment, configuration, cases, and authorised environment. Findings do not certify a component vendor, establish universal safety, or approve release. Unexamined provider paths and specialist questions remain explicit limitations.
Ways to engage
Related engagement scopes
AI Agent Evaluation Evidence Diagnostic
Find the gaps between your release decision and the evidence behind it. Start with one agent workflow and a bounded set of questions.
AI Agent Evaluation Foundation
Turn scattered tests into a repeatable release discipline. Connect the behaviours that matter to datasets, evaluation criteria and evidence your team can maintain.
AI Agent Component Evaluation
Find where agent behaviour breaks down. Follow memory, retrieval and delegation through your workflow, then isolate the components that need attention.
Continue exploring
Related insights and resources
Engineer the harness, not just the prompt
Define the contracts around an agent's decisions: what state it trusts, which actions can execute, how interrupted work resumes, and what evidence an operator needs to recover safely.
What would justify releasing this agent?
Replace a single pass rate with a reviewable chain of requirements, cases, observations and judgements. Distinguish task quality, control failures and uncertainty before increasing an agent's exposure.
The evidence behind the go-live decision
A working paper for leaders who need to explain why an agent workflow is ready for its next release. Includes a proposed evidence-pack structure, decision table and review agenda.