What belongs in an AI agent release evidence pack?

An AI agent release pack is not a celebratory report showing that the latest build scored better. It is the smallest complete body of evidence an accountable owner needs to decide whether a defined agent configuration may perform a defined job under defined controls.

That distinction matters because the result is produced by a system, not just a model. Prompts, retrieval, tool descriptions, permissions, orchestration, user interfaces and human review can all change what the agent does. The pack must keep those conditions attached to the findings. Otherwise, an apparently impressive result cannot answer the practical question: is this the system we intend to release, and what exactly are we accepting?

Begin with the decision, not the test run

The cover sheet should state the proposed decision in plain language: release, release to a restricted cohort, continue a controlled pilot, remediate and retest, or stop. It should name the decision owner, the version being considered, the intended workflow, the permitted users and the operating environment.

This is also where the team records consequence. An agent that drafts an internal note and an agent that changes a customer’s account should not cross the same evidence threshold. The NIST AI Risk Management Framework treats risk as a combination of likelihood and consequence and calls for measurement that informs management action. The release question therefore needs both parts: how often did an unwanted behaviour occur, and what could that behaviour do here?

A one-page decision frame can use six fields:

Field	Release question
Intended use	Which workflow outcome may the agent pursue?
Authority	Which data, tools and actions may it access?
Exclusions	Which users, cases or decisions remain outside scope?
Change	What is different from the last accepted version?
Evidence threshold	What must be demonstrated for this release?
Decision owner	Who can accept, constrain or reject the release?

Freeze the system that produced the evidence

The pack needs a configuration manifest, ideally generated rather than typed. Record model and provider version, system and developer prompts, tool schemas, policy rules, retrieval index, memory behaviour, runtime settings, application commit, evaluation harness commit and dependency versions. Add the identities and permission scopes used during testing, without copying secrets into the pack.

This is ordinary configuration discipline applied to an unusually changeable system. The UK NCSC secure AI development guidance recommends documenting models, datasets and prompts, tracking assets and restoring to a known good state. A release result without this manifest is historical trivia: it cannot reliably be reproduced, compared or investigated.

Record the external dependencies too. If a hosted model or search API may change behind a stable name, say how the deployed system pins, detects or responds to that change. If no pinning mechanism exists, the release decision should include the resulting monitoring and rollback obligation.

Show what was tested and why it represents the workflow

The evidence index should map each important requirement or risk to cases, measures and results. Do not start with aggregate accuracy. Start with coverage.

Useful case families include routine work, legitimate edge cases, prohibited requests, ambiguous instructions, stale or conflicting context, tool faults, interrupted workflows, hostile content and handoffs to people. State where the cases came from: sampled historical work, synthetic construction, an incident, a policy rule or adversarial research. Record selection rules and exclusions so a reviewer can judge representation rather than assume it.

NIST’s ARIA pilot evaluation combined model testing, red teaming and field testing because each level reveals different evidence. That is a useful release-pack pattern. A harness can provide repeatability, adversarial testing can search for plausible failure, and controlled user work can expose interaction effects neither predicts alone.

A simple coverage matrix makes omissions visible:

Requirement	Routine	Boundary	Adversarial	Fault	Field observation
Correct workflow outcome	✓	✓		✓	✓
Authorised tool use	✓	✓	✓	✓	✓
Required confirmation	✓	✓	✓		✓
Safe recovery		✓	✓	✓	✓

Blank cells are not automatically defects. They are prompts for an explicit decision about whether the missing evidence matters.

Preserve actions, states and repeated behaviour

For an agent, the final message is only one observation. The pack should link representative traces showing the initial state, context presented, tool requests, authorisation decisions, tool responses, state changes, retries, final state and human handoff. Sensitive traces can remain in controlled storage; the decision record needs stable identifiers and a sanitised description.

State-based grading is especially important. The tau-bench research evaluates tool-using agents by comparing the final database state with the intended state, and repeats tasks to test consistent success. The practical lesson is not to copy its domain or thresholds. It is to check what actually changed and to repeat non-deterministic cases enough to expose variance.

Report counts and distributions, not only averages. Separate task completion from policy compliance. A system that completes a refund but skips required confirmation did not produce a qualified success. Also show invalid or inconclusive runs, grader disagreement and infrastructure failures rather than silently dropping them.

Turn observations into findings and release conditions

Every material finding should connect five things: the observed behaviour, the requirement it affects, plausible consequence, reproducibility and treatment. A screenshot of a surprising answer is not yet a finding. Nor is every odd response a release blocker.

Use a disposition table that keeps severity and decision separate:

Finding	Evidence	Treatment	Release effect	Owner
Confirmation omitted on one policy path	Case IDs and traces	Fix policy routing; regression set	Block until retest	Product engineering
Handoff summary loses optional note	Human rubric sample	Backlog and monitor	Accept for restricted release	Workflow owner
Search dependency timed out	Fault injection runs	Retry once, then hand off	Release with control	Platform operations

The NIST generative AI profile cautions that red-team outputs require further analysis before they become governance decisions. The same applies to all evaluation findings. Translate them into requirements, controls, acceptance criteria or explicitly accepted residual risk.

Include the controls that make the result operable

Pre-release evidence expires the moment operating conditions diverge. The pack should therefore describe production safeguards: permission enforcement, approval gates, rate and cost limits, isolation, logging, monitoring, incident response, rollback and shutdown. For each control, state its owner, trigger and evidence that it was tested.

Add post-release obligations rather than a vague promise to monitor. For example:

• rerun the release suite after any model, prompt, tool, policy or retrieval change;
• sample completed transactions weekly for the first month;
• page an operator only for defined, actionable loss of control;
• add confirmed incidents to the regression set within an agreed interval;
• suspend a capability when a stated threshold is crossed.

These obligations connect a one-time gate to a managed lifecycle. ISO/IEC 42001 describes an AI management system in terms of policies, objectives and processes for responsible development or use, including continual improvement. The release pack is not proof of conformity to that standard; it is a practical place to make the local process visible.

Give the reviewer a clean decision surface

Keep raw logs, datasets and scripts in versioned annexes. The main pack should be navigable in one review: decision frame, configuration, evidence coverage, result summary, material findings, residual risk, controls, operating obligations and sign-off. Each summary statement should lead to underlying evidence through a stable link.

End with an explicit record: decision, scope, conditions, expiry or review trigger, dissent, open actions and accountable owner. “Passed evaluation” is not a decision. “Approved for 10 trained users to draft cases, with all external actions disabled, pending a 30-day review” is.

The pack does not prove that the agent will behave correctly in every unseen situation. Its job is narrower and more valuable: make the evidence boundary, unresolved uncertainty and ownership clear enough that a release decision can be challenged, reproduced and revisited.