The anatomy of a tool-using AI agent

A product diagram that puts a large language model in the middle and draws arrows to “memory” and “tools” hides the decisions that matter. The model may propose an action, but another component supplies credentials, invokes the tool, records the result and decides whether work may continue. If those responsibilities are not explicit, a team cannot say where authority is enforced, which state is authoritative or what evidence would justify release.

The practical architecture question is therefore not “Which model powers the agent?” It is: what chain turns an instruction into a real-world effect, and where can that chain be inspected or stopped?

Start with an action path, not a component catalogue

Consider an agent asked to reschedule a customer delivery. The successful path might be:

1. identify the customer and requested delivery;
2. retrieve the current order and applicable policy;
3. determine whether the change is allowed;
4. ask for confirmation if the new date changes price or service level;
5. call the fulfilment system with a constrained credential;
6. verify the resulting order state; and
7. tell the customer what happened, including any unresolved exception.

That sequence immediately reveals more than a generic agent diagram. The customer message is an input, not permission. Retrieved policy is evidence, not an instruction source. The proposed change is not complete until the business system confirms it. A confirmation gate has to bind to the exact action that will occur. A retry must not create a second change.

NIST’s tool-use taxonomy distinguishes perception, reasoning, resource management and action tools, and also separates read-only, constrained-write and write access. That is a useful starting vocabulary because two tools with similar names can create radically different consequences.

Eight layers make the system reviewable

The architecture can be mapped as eight cooperating layers. They need not be separate services, but their responsibilities should be separable in design and evidence.

Layer	Question it must answer	Evidence worth retaining
Intent	What outcome did the person or upstream system request?	Original request, interpreted goal, scope and confirmation
Context	What information is available for this decision?	Retrieved records, provenance, versions and access basis
Planning	What sequence of steps is proposed?	Plan, alternatives, stopping conditions and revisions
Tools	What operations can affect the environment?	Tool contract, arguments, responses and error semantics
Authority	Which identity may perform each operation under which limits?	Principal, grant, policy decision and approval binding
State	What has happened, what is pending and what is final?	Durable transitions, deduplication keys and checkpoints
Verification	Did the intended outcome occur without prohibited effects?	End-state checks, policy checks and human review
Operations	How is the system observed, changed, paused and recovered?	Traces, version manifest, incidents, rollback and ownership

The language model participates mainly in interpretation and planning. It may also help select context or evaluate a result, but it should not quietly become the authoritative source for identity, policy, business state or completion.

Academic surveys describe planning, memory and tool use as recurring agent components, while also noting that architectures vary substantially (Frontiers of Computer Science). The eight-layer model is not a claim that every agent must use one framework. It is a review lens for locating responsibilities that exist whether or not the implementation names them.

Context and authority must travel separately

An agent may know that a customer is eligible for a refund without being allowed to issue one. It may retrieve an administrator’s instructions without inheriting the administrator’s authority. Keeping context and authority separate prevents a recurring design error: turning information visible to the model into permission available to the runtime.

For every tool call, record at least four identities:

• the human or system that initiated the workflow;
• the agent instance proposing the action;
• the credential or service principal executing it; and
• the owner accountable for the policy that allowed it.

Then bind any approval to the intended target, parameters, amount or scope, and expiry. “Approved” in conversation history is too ambiguous when the plan can change after approval. The enforcement point should receive a structured authorisation decision, not infer one from prose.

Tools are contracts with failure semantics

A tool definition needs more than a name, description and JSON schema. A consequential tool contract should state:

• whether the operation reads or changes state;
• whether it is reversible, idempotent or compensatable;
• which identity and permission it requires;
• how partial success appears;
• which errors are safe to retry;
• what authoritative record confirms completion; and
• what data may enter or leave the boundary.

This is where ordinary distributed-systems engineering becomes central to agent reliability. A timeout does not prove that an operation failed. A syntactically valid response does not prove that the desired state exists. A model deciding “try again” is not a recovery policy.

Research on evaluating reasoning and action argues that tool-using agents must be assessed across their combined reasoning and execution behaviour, rather than scoring the language response alone (AgentBoard research). The most useful test oracle is often the state of the external system after the tool call.

State turns a conversation into a workflow

Conversation history is useful evidence, but it is a poor workflow database. Durable state should identify the workflow instance, current step, completed effects, pending approvals, active leases, retries and terminal outcome. State transitions should follow declared rules—for example, proposed → approved → executing → verified—with invalid transitions rejected.

This gives recovery a dependable starting point. After a crash, the runtime can check whether the fulfilment change already happened, rather than asking a model to reconstruct events from a transcript. It also enables a reviewer to distinguish “the model said it finished” from “the system verified the intended state.”

Memory is different again. Remembered preferences or prior episodes may inform a future decision, but they should not overwrite transaction state. If a customer once preferred Tuesday delivery, that memory is context; the current confirmed order date remains the business record.

Verification closes the loop

The agent should verify effects using a source that is independent of its own claim. For a database update, read the committed record. For a message, retain the provider’s accepted identifier and delivery status where available. For a code change, run specified tests and inspect the repository diff. For a human handoff, require acknowledgement from the receiving queue or person.

Verification also checks negative requirements: no unapproved tool was used, no protected data crossed the boundary, and the action stayed within amount and time limits. The NIST Generative AI Profile recommends documenting testing, limitations and risk-management actions across the lifecycle. Applied here, that means the release record should connect each material action class to a tested control and observable result.

Use the map to make a release decision

Before a tool-using agent receives production authority, walk one representative workflow and one adverse workflow through all eight layers. For each boundary, ask:

• Is the authoritative input named and versioned?
• Is authority enforced outside model-generated text?
• Can a duplicate, late or reordered message cause a second effect?
• Can the runtime distinguish unknown, pending, failed and verified outcomes?
• Is there a safe stop and a named human owner?
• Can evidence connect the request to the final business state?

If a layer has no accountable implementation or observable evidence, narrow the agent’s authority until it does. The model can still draft, recommend or collect information while the missing control is built. That is a legitimate product boundary, not a failure of ambition.

The durable design principle is simple: the model may help decide what to do, but the surrounding system must determine what it can do, record what it did, and verify what actually changed.