How to map an AI agent’s attack surface

An agent is not one attack surface. It is a chain of surfaces joined by a model that decides what to do next. A browser retrieves hostile content; retrieval exposes internal documents; memory carries old instructions forward; an orchestrator selects tools; credentials authorise actions; and an execution environment reaches systems that may never appear in the model diagram.

A useful map makes those paths visible enough to answer a release question: which attacker can influence which component, what authority could that influence acquire, and which control prevents the unacceptable outcome?

Fix the scope around one business workflow

Start with a named workflow and a concrete release boundary, not “our AI platform.” For example: an agent reads support tickets, retrieves customer and product information, drafts responses, issues refunds below a limit and escalates exceptions. Name the business owner, environments, user populations and systems of record. Record what is out of scope rather than allowing the boundary to drift during review.

OWASP describes threat modelling through four questions: what are we working on, what can go wrong, what will we do about it, and did we do a good enough job. The first question is operational. A diagram with no version, owner or workflow decision quickly becomes decorative.

Set three to five unacceptable outcomes before drawing components. Examples include cross-customer disclosure, an unauthorised refund, persistence of malicious instructions, arbitrary code execution, or loss of the ability to halt the system. These outcomes keep the exercise focused on business impact rather than an unranked catalogue of threats.

Draw six kinds of node and every crossing

Use ordinary data-flow notation, then add agent-specific meaning. NIST’s data-centric threat-modeling draft emphasizes understanding selected data, authorised locations and flows, and threats against it. An agent map should contain:

1. Actors: users, administrators, attackers, reviewers, other agents and external services.
2. Decision components: model endpoints, planners, routers, graders, policy engines and approval services.
3. Data stores: retrieval indexes, memories, traces, credentials, queues and systems of record.
4. Interfaces and tools: APIs, MCP servers, browsers, code execution, messaging and file operations.
5. Execution zones: user device, application service, sandbox, cloud account, customer environment and third-party boundary.
6. Controls: authentication, authorisation, validation, isolation, egress policy, monitoring and emergency stop.

Draw directed flows for data, instructions, credentials and effects. Mark every place trust changes: public web to browser, customer tenant to shared retrieval, model output to tool argument, agent identity to downstream API, sandbox to network, and generated proposal to human approval.

Annotate the map with authority

Conventional diagrams frequently show an API call but omit whose authority it uses. Add the principal, credential source, scopes, resource boundary, lifetime and approval condition to every effectful edge.

Edge	Identity question	Authority question	Failure to test
Orchestrator → refund tool	User, agent, or shared service?	Which customers, amounts and reasons?	Cross-tenant or over-limit refund
Browser → internet	Which workload instance?	Which destinations and methods?	Data sent to attacker domain
Agent → memory	Which user and task?	Who can read, alter or delete it later?	Poisoning or cross-user leakage
MCP client → server	Which client and resource owner?	Is the token intended for this server and tool?	Token misuse or confused deputy
Operator → kill switch	Which responder role?	Can it revoke tools and queued work?	Apparent stop while actions continue

This annotation often reveals “ambient authority”: a general service credential mounted into the process, a token reused across tools, or a human account silently delegated to a model. Those are design findings before an attacker arrives.

Trace instruction provenance and durable state

Colour or label inputs by provenance: developer policy, authenticated user instruction, organization-controlled data, third-party data, and attacker-controlled content. Do not assume retrieved documents are trusted because they are inside an enterprise index; external uploads and compromised sources may already be present.

Then follow what becomes durable. Memory, summaries, cached tool results and generated tasks can preserve malicious influence after the original content disappears. Ask who can write each store, how entries are associated with tenant and identity, whether provenance survives summarization, and how an operator can inspect and delete affected items.

The ENISA AI cybersecurity landscape uses a lifecycle view and highlights assets and supply-chain relationships as prerequisites for threat analysis. Apply that thinking beyond model training: prompt templates, evaluation datasets, tool packages, container images, protocol servers and hosted model endpoints all enter the operational supply chain.

Turn unacceptable outcomes into attack paths

For each outcome, write at least one path as a sequence of preconditions. A cross-customer disclosure path might be:

attacker submits ticket → content influences retrieval query → shared index returns another tenant’s document → response tool sends it to attacker.

Now challenge every arrow. Can the attacker control the ticket format? Does the retrieval service enforce tenant filtering independently of the generated query? Does the response tool validate that attachments belong to the authenticated case? Is the final destination fixed or model-supplied?

Use established categories such as spoofing, tampering, repudiation, information disclosure, denial of service and elevation of privilege to prompt coverage, then add agent-specific paths: indirect prompt injection, tool-description manipulation, memory poisoning, excessive agency, unsafe inter-agent delegation and resource exhaustion. OWASP’s agentic threat guidance can seed questions, but only the local map shows which paths are feasible.

Rank paths by consequence, exposure and control strength. Avoid pretending uncertain probabilities are precise. A low-frequency path to a catastrophic and irreversible action may still justify elimination or a mandatory human gate.

Attach controls and evidence to the same map

Every material attack path needs a response: eliminate the capability, constrain it, detect and contain it, transfer a defined risk, or record accountable acceptance. Put the control on the edge where it is enforced. “The agent checks policy” is not a boundary if the same model can ignore the check. “The refund API rejects amounts above the signed mandate” is testable.

For each control, specify evidence:

• configuration or policy as deployed;
• an automated test that attempts the prohibited path;
• a trace showing the decision and actual end state;
• monitoring for bypass or degradation; and
• an owner and review trigger.

The NCSC secure-design guidance advises teams to model threats, design for security as well as functionality, and restrict actions triggered by AI components. A control map makes that advice releaseable: the review can distinguish planned, implemented and verified controls.

Validate the map against the running system

Architecture workshops capture intended design. Security testing must find the effective design. Inventory actual endpoints, DNS destinations, mounted secrets, cloud permissions, tool schemas, model versions and message queues. Compare traces from representative tasks with the drawn flows. Unexpected network traffic or a credential absent from the diagram is a finding, even before exploitation.

Run one end-to-end test for every high-priority path. Inject hostile content through the real ingestion route, use canary records for protected data, and observe the actual downstream state. Fault authorisation services, revoke credentials mid-task, poison a memory item, change a tool description, and halt the agent while actions are queued. Record whether the control failed safely and whether the evidence is sufficient to explain the result.

The map is complete enough for release when owners can point from each unacceptable outcome to its plausible paths, enforced controls, tests and residual decision. It will never be permanently complete. Treat every new integration as a new crossing, every incident as a correction, and every apparently helpful capability as authority that must appear on the page.