What public-sector buyers should request from an agent supplier

An agent supplier demonstrates a polished public-service interaction: the system understands a request, finds information and completes a task in seconds. The buyer asks about model accuracy and receives a benchmark slide. Neither the demo nor the score establishes how the service handles a disputed record, inaccessible channel, malicious document, model change or failed external action.

Public-sector buyers need evidence about the complete service and enforceable rights across its life. The request should describe the public outcome, operating boundary and accountability—not merely ask vendors to provide “an AI agent.”

Procure the service outcome and boundary

State the problem in operational terms. Who uses the service? What decision or action is being supported? Which outcomes matter? What must remain available for people who cannot or should not use the automated route?

Define the agent’s proposed authority:

• information it may read;
• decisions it may recommend;
• systems and tools it may call;
• actions it may take;
• conditions requiring human approval;
• cases it must refuse or escalate; and
• people who can challenge or correct an outcome.

The UK guidelines for AI procurement advise buyers to begin with the service need, assess data and risk, avoid lock-in and plan lifecycle management. That is more useful than specifying a fashionable architecture before understanding the public service.

Request an evidence pack, not a performance headline

Require evidence that matches the intended population, workflow and consequence.

Evidence area	What to request
Scope	supported and excluded cases, known assumptions, authority matrix
Evaluation	task design, datasets, metrics, severe failures and subgroup results
Actions	tool schemas, permission enforcement, state confirmation and rollback
Human oversight	reviewer role, information shown, override and escalation tests
Security	threat model, testing, incident history and remediation process
Privacy	data flow, purposes, retention, model use and deletion evidence
Accessibility	user research, standards, assisted routes and test results
Operations	monitoring, change control, resilience, support and recovery exercises

Ask for failed tests and unresolved risks, not only averages. Determine who performed the evaluation and whether the buyer can reproduce critical scenarios in a controlled environment. NIST’s AI RMF calls for test sets, metrics and tools to be documented and for performance criteria to be measured in conditions similar to deployment (AI RMF Core).

Make data terms technically specific

Map every class of public, personal, sensitive and operational data. Ask where it is processed, which subprocessors receive it, how long each copy persists and whether prompts, outputs or feedback are used to train any model. Define deletion, return, breach notification and audit evidence.

Require retrieval to respect the source system’s permissions and retention rules. Test cross-user leakage and unauthorised inference. If synthetic data is used in evaluation, require evidence that it represents the cases and groups the service must handle without exposing real people.

The UK Data and AI Ethics Framework applies principles of privacy, fairness, safety and transparency to public-sector development, procurement and use. Translate those principles into observable supplier obligations rather than accepting a policy statement.

Retain control over change

An agent can change when the model, prompt, retrieval index, tool, policy, safety layer or provider changes. Require a component inventory, version record and definition of material change. The contract should specify notice, regression evidence, buyer testing, emergency changes and rollback.

Do not accept a service whose supplier can materially alter behaviour without telling the service owner. Ask how older versions and incident evidence remain available for investigation.

UK procurement policy on AI transparency asks in-scope organisations to understand and manage AI use in commercial activity, including supplier use of AI and protection of confidential information (PPN 017). The same transparency should continue after award.

Contract for incidents, continuity and exit

Specify what constitutes an AI or agent incident: harmful output, unauthorised action, data exposure, loss of traceability, material performance deterioration or dependency failure. Require notification times, evidence preservation, joint investigation, corrective action and communication responsibilities.

Test continuity before go-live. Can the service reduce the agent to read-only, route work to people, switch off one tool or operate through a fallback? Who has the technical ability to do so?

Plan exit while procurement leverage exists. Require exportable records, configurations, evaluation cases and decision logs in usable formats; verified deletion; transition support; and a route to continue the public service without the supplier. OECD guidance highlights contract treatment of data, privacy, lock-in, ongoing testing, performance and lifecycle obligations (OECD).

Evaluate the supplier in a controlled trial

Give qualified bidders the same representative scenarios, data conditions and action boundaries. Include ordinary cases, exclusions, inaccessible journeys, conflicting records, untrusted content, tool outages and uncertain transaction results.

Score evidence quality and behaviour, not presentation. A practical weighting can examine:

• valid task outcome and severe failure rate;
• transparency and reproducibility of evidence;
• authority and security control effectiveness;
• accessibility and human escalation;
• integration and operational resilience;
• change, incident and exit capability; and
• total operating cost, including review and correction.

Keep evaluators multidisciplinary and record judgement. Do not let a procurement score imply a legal or compliance conclusion that the process did not establish.

Ask suppliers to explain one difficult case

Use a scenario with an incomplete public record, a person who needs an assisted channel, a malicious attachment and an external action that times out. Ask each bidder to demonstrate the full trace: what the agent knew, what it refused, which human role intervened and how the final state was reconciled. Require the supplier to identify residual uncertainty rather than award points for uninterrupted automation.

The explanation should be understandable to service, commercial, security and data specialists without exposing protected intellectual property. If the buyer cannot obtain enough evidence to investigate a harmful decision or reproduce an acceptance test, that is a commercial and operational constraint to score—not a detail to postpone until implementation.

Make acceptance the start of assurance

Define acceptance criteria before award and connect them to payment or rollout milestones where appropriate. Begin with limited users, cases and authority. Monitor outcomes, complaints, overrides, correction, accessibility and incidents. Retain the right to re-test after material change and to reduce authority when evidence deteriorates.

Name the buyer-side owner for each obligation and make evidence delivery operational. A contract clause requiring quarterly testing has little value if no team can access the environment, interpret the results or require remediation. Schedule service reviews around public outcomes and control evidence, not only uptime and ticket counts.

Include service users and frontline staff in those reviews through appropriate, privacy-respecting mechanisms. Complaints, assisted-channel demand and repeated manual correction may reveal a failure that system availability misses. Require the supplier to support investigation without treating every request for evidence as a paid customization.

This checklist is technical and commercial analysis, not procurement, legal, data-protection or regulatory advice. Public bodies should adapt it with the responsible specialists and applicable rules.

The best supplier response will not claim that the agent eliminates uncertainty. It will show where the system works, where it stops, what evidence supports those boundaries and how the buyer remains in control when technology or public need changes.