Softlander Book a call
Agent security and cybersecurity

Retrieval poisoning tests

Test how corrupted retrieval sources influence agent outputs and actions

New documents or index updates feed a consequential workflow. The decision is how to test how corrupted retrieval sources influence agent outputs and actions. The work begins with your actual workflow and looks for the evidence that separates intended behaviour from a consequential exception. It is particularly relevant to search platform owners and application security leads. A useful outcome is an explicit account of what was examined, what remains uncertain, and which decision the evidence can support.

Book a 30 minute readiness callBrowse this capability group

The agent security relationship

  1. Starting evidenceCorpus permissions

    Establish the basis for corpus permission.

  2. Decision boundaryRanking influence

    Vary content and metadata in controlled fixtures.

  3. Decision artefactRetrieval poisoning case set

    Retain the scoped finding and its unresolved dependencies.

Proposed working model. A bounded challenge cannot establish exhaustive protection.

Decision

Frame the decision around corpus permission

Questions and the evidence they need

Criterion / evidenceCorpus permission
Inspect corpus permissions for the relevant constraint or observation.
Criterion / evidenceRanking influence
Inspect indexing rules for the relevant constraint or observation.
Criterion / evidenceGrounded action
Inspect ranking behaviour for the relevant constraint or observation.
Read each criterion against the available evidence before deciding that additional work is needed.

The decision is how to test how corrupted retrieval sources influence agent outputs and actions. This is relevant to search platform owners and application security leads when new documents or index updates feed a consequential workflow. Name the affected workflow, the consequence of a wrong decision, and the owner who can act on the result. The initial discussion should establish which evidence is already available and which uncertainty is material enough to investigate.

The specific failure hypothesis is that a manipulated source is retrieved as authoritative and overrides the actual business policy. It remains a hypothesis until examined in the deployment. Existing internal tests or operating records may already resolve it; identify the uncovered question before commissioning additional work.

Mechanism

Retrieval poisoning can enter through content or metadata

Relationships within agent security

Source or conditionCorpus permission
Identify who can alter indexed material.
Crossing or choiceRanking influence
Vary content and metadata in controlled fixtures.
Consequence or decisionGrounded action
Compare selected evidence with the resulting operation.
Observe the control and resulting business effect. Each relationship needs evidence in the actual scope.

What this specialty examines

Retrieval poisoning examines source admission and influence on answers or actions. Inspect corpus permissions, indexing, metadata and source selection. A well-formed citation can point to an unreliable document, so citation presence and source trust are separate questions.

The review should use a controlled corpus to vary source content, metadata and relevance while preserving expected outcomes. Identify the normal route, relevant exceptions and the authoritative record of the outcome. Inspect the relevant business-system record, enforcement event or process evidence alongside any reported status. Keep incomplete cases visible, with an explanation of what their absence prevents the owner from concluding.

The tradeoff to make explicit

For this decision, open contribution improves coverage while source authority and review need explicit treatment. Compare the available design or operating choices against the same intended outcome. If one option removes a required control or changes the task boundary, describe that difference before treating its apparent simplicity, speed or lower effort as an improvement. The decision should expose the tradeoff instead of burying it in an aggregate score.

Preparation

Connect the working inputs to the boundary being examined

Evidence packet for corpus permission

Illustrative working format

BasisCorpus permissions
Specify the subject, operation, resource, and context at the enforcing boundary.
ContextIndexing rules
Confirm the applicable revision and who resolves disputed interpretation.
ObservationRanking behaviour
Name the custodian and the conditions requiring a refreshed version.
DependencySource-version records
Match this description to the deployed or proposed boundary being examined.
Illustrative preparation record. Availability, revision, and handling need confirmation.

Start with corpus permissions, indexing rules, ranking behaviour and source-version records. Identify the owner, revision and operating context of each source. Sanitised examples may support initial design, but differences from actual conditions should remain explicit. Where direct access is unsuitable, a customer-controlled evidence or execution route may be considered, with agreed handling and retention.

The scope also needs the security decision owner, agreed expectations for a representative threat scenario, and the authority to inspect the relevant boundary. Specify what evidence would distinguish the failure in which a manipulated source is retrieved as authoritative and overrides the actual business policy from acceptable behaviour. If stakeholders disagree about intended behaviour, resolve the interpretation before classifying the observed difference as a defect.

How the work proceeds

Carry corpus permission through to an accountable finding

From the scoped material to a decision record

  1. Establish conditionsCorpus permissions

    Define the hypothetical attacker’s starting influence.

  2. Examine the relationshipRanking influence

    Apply the scoped check to grounded action.

  3. Retain the findingRetrieval poisoning case set

    Record the evidence, open dependencies, and the owner’s next decision.

Proposed sequence; resolve interpretation or access gaps before relying on the next step.

Define the controlled threat scenario

Review corpus permissions, indexing rules, ranking behaviour and source-version records with search platform owners and application security leads. State what the hypothetical attacker controls, the protected action and the systems included in the authorised exercise. Prepare synthetic records and harmless markers where appropriate, with stopping conditions for unexpected effects. The preparation output is a scenario brief that distinguishes an influence attempt from a prohibited action actually reaching a business resource.

Challenge the path and review effects

Use a controlled collection with conflicting or manipulated records. Observe which source is selected and how authority, freshness and contradictions are handled. Preserve source versions so a later index change does not obscure the tested condition.

Select a control response and retest

Build a retrieval poisoning case set with ingestion and grounding findings from the reviewed observations. Separate agent behaviour, application enforcement and the resulting resource state so the recommended change reaches the responsible layer. Choose a bounded retest that reproduces the original condition and checks legitimate behaviour after remediation. If the required investigation exceeds the agreed technique or expertise, retain the unresolved path and identify the qualified review needed.

Indicative artefact

Make the retrieval poisoning case set usable by its owner

Retrieval poisoning case set

Illustrative working format

Annotated fieldRetrieval poisoning case set
Identify the assessed workflow, revision, and decision boundary.
Annotated fieldIngestion
Link supporting and contrary observations to inspectable evidence.
Annotated fieldGrounding findings
Link observations and counterevidence to the case, configuration, and conclusion limit.
Illustrative artefact structure, not a completed engagement or an observed result.

A useful decision artefact is a retrieval poisoning case set with ingestion and grounding findings. Supporting material may include selected cases, configuration or process records, evidence references, a disagreement log and prioritised actions. Retain the case selection rationale and material disagreements alongside the result; otherwise the result may be mistaken for complete coverage.

Illustrative scenario

Examine grounded action through a constructed exception

A constructed case and its decision paths

Illustrative scenario

  1. SituationCorpus permission

    A manipulated source is ranked above the correct record in a test corpus.

  2. Question to examineRanking influence

    Hold task expectations fixed while changing controlled corpus metadata.

  3. Conditional next actionGrounded action

    Locate whether ingestion, ranking, or source interpretation changed the outcome.

Fictional example. No executed test, customer engagement, or measured outcome is implied.

Consider a fictional situation: a fictional policy index contains a plausible obsolete document that encourages unauthorised account changes. The scenario is illustrative, with no client engagement, executed test or measured improvement implied.

For the obsolete policy, compare it with the current approved record. The agent may need to surface the conflict or use an authoritative source rule. The evidence should distinguish retrieval ranking problems from an interpretation that ignores source status.

Scope and next action

Keep the conclusion within the agreed threat scenario

Confirm fit before the next commitment

Included boundaryCorpus permission
This review is limited to the agreed threat scenario.
PrerequisiteCorpus permissions
The relevant owner must confirm access and the intended operating conditions.
Claim limitRetrieval poisoning case set
A bounded challenge cannot establish exhaustive protection.
Bring the triggering change and existing evidence; resolve the dependency that could alter the decision.

Authorised scenario testing is not a claim of exhaustive penetration testing or guaranteed protection. Intrusive security work requires qualified specialists and a separately agreed scope. Any engagement depends on an agreed scope, qualified team, authorised access and confirmed availability. Bring the affected workflow, its current configuration or process, the triggering change and the evidence already available. The immediate scope question is prompted by the situation in which new documents or index updates feed a consequential workflow; confirm which missing observation would change the owner’s next action.

Ways to engage

Related engagement scopes

Agent Release Readiness Review

The Agent Release Readiness Review is a fixed-scope, three-week independent test of one AI agent workflow that shows a release board what the agent does, what it is allowed to do and what happens when it is wrong, and ends in a signed recommendation.

3 weeksRead about this offer

AI Agent Runtime Security Review

Make every delegation explicit. Examine who the agent acts for, where it can execute and which tools it can reach when instructions alone are not enough.

Read about this offer

AI Agent Red Team

Challenge the boundaries that matter to your workflow. Test agreed attack scenarios against agent instructions, tools and data access.

Read about this offer

Continue exploring

Related insights and resources

Agent security6 MIN READ

The permission boundary behind every agent action

An agent can only be evaluated in the context of what its tools allow. Examine identity, delegated authority and action boundaries before you put a workflow in motion.

Whitepaper5 MIN READ

Before your agent can act, define its authority

A working paper for the moment an AI system moves from advice to action. Map identity, permissions, approvals and recovery into an authority model your team can review and test.

Agent security6 MIN READ

Running an AI-agent incident tabletop exercise

A useful tabletop follows one plausible agent failure through detection, containment, evidence, business recovery and accountable return to service.

A little more clarity

Questions to consider.

Start with the questions that shape a better decision.

Is a citation enough when the cited source has been poisoned?

No. A citation identifies a source, but the source may be manipulated, obsolete or inappropriate for the decision. Evaluate source authority and how conflicts are handled alongside citation behaviour.

Which corpus records establish source authority?

Start with corpus permissions, indexing rules, ranking behaviour and source-version records. Record their revision, owner and relationship to the intended workflow. Missing context should be identified before interpreting outcomes; a sanitised fixture is useful only when its relevant differences from the operating case are understood.

How should conflicting or manipulated retrieval sources be tested?

Use a controlled corpus to vary source content, metadata and relevance while preserving expected outcomes. State the hypothetical attacker's control and inspect attempted actions, tool enforcement and actual effects separately. A refusal, a blocked call and a successful prohibited action are different results. Preserve unsuccessful challenges as well as adverse findings.

What belongs in a retrieval poisoning case set?

The proposed output is a retrieval poisoning case set with ingestion and grounding findings. Include the selected conditions, supporting observations, unresolved differences and next decision. A reviewer should be able to follow each important conclusion back to its evidence and understand the limits without depending on an undocumented explanation from the author.

When is a citation insufficient grounding?

A citation is insufficient when its source is obsolete, manipulated, unauthorised or unsuitable for the decision. Evaluate source authority and conflicting evidence rather than equating attribution with correctness.

What if existing evidence already answers the question?

Review its scope, provenance and configuration before proposing new work. Existing work from search platform owners and application security leads may already be sufficient. In that case, capture the decision, unresolved limitations and the change that would require another review. Additional assessment should address a real gap.

What would make the findings inconclusive?

Missing corpus permissions, indexing rules, ranking behaviour and source-version records can prevent a reliable interpretation, depending on the case. Record the specific missing observation and its implication. The owner can supply evidence, narrow the claim or defer the dependent decision; an incomplete case should not silently be counted as a pass.

Start with the decision you need to make.

Tell us about the agent workflow you want reviewed, and where you need greater clarity. Scope and availability are confirmed together.

Book a 30 minute readiness call Get the agent release checklist

Connected disciplines

Data-exfiltration testing

Validate whether information can leave the intended agent boundary. The review connects data classifications, outbound tools, recipient controls and synthetic canary records to a bounded review of agent security, giving the security decision owner a clearer basis for the next action.

Agent supply-chain security

Review the chain of components that supplies an agent deployment. The review connects dependency inventories, provenance records, update paths and approval policies to a bounded review of agent security, giving the security decision owner a clearer basis for the next action.

Model and dependency security

Assess security-relevant changes in model and software dependencies. The review connects version manifests, dependency advisories, integration tests and provider configuration to a bounded review of agent security, giving the security decision owner a clearer basis for the next action.