The future of AI assurance is continuous—but not fully automatic

A release assessment captures one configuration under selected conditions. An agentic system changes the moment its model, prompt, retrieval corpus, tools, permissions, users or environment changes. Even without a deployment, the mix of tasks can drift and external content can become adversarial.

Assurance therefore has to continue after release. But “continuous” should not mean an automated score silently approves whatever the system does next. Automation is well suited to collecting evidence, replaying tests and detecting deviations. It is less suited to deciding which harms matter, whether evidence is sufficient, who may accept residual risk or how competing interests should be balanced.

The future operating model is continuous evidence with accountable decisions.

Move from a certificate moment to an evidence loop

A static approval asks whether the tested version met a threshold. A continuous assurance loop asks four recurring questions:

1. Is the deployed system still the version that was approved?
2. Is it operating inside the tested conditions and authority envelope?
3. Are outcomes and failure patterns changing?
4. Does new evidence require narrower authority, remediation or renewed review?

The NIST AI RMF Core states that AI risk management should be continuous and timely across the lifecycle. It calls for production monitoring, risk tracking over time, regular evaluation, post-deployment monitoring, incident response, recovery and change management. The Generative AI Profile extends that risk-management approach to generative AI.

This is not a promise of perpetual correctness. It is a disciplined way to discover when the original evidence no longer describes the deployed system.

Build the loop around decisions, not dashboards

Continuous assurance can produce endless metrics without changing an action. Start by defining decisions and triggers.

Decision	Evidence inputs	Example trigger
Continue within current authority	outcome quality, control health, incident rate	all critical thresholds remain inside tolerance
Investigate	drift, anomalies, complaints, uncertain verdicts	sustained change or material outlier
Narrow or pause	prohibited effect, failed control, unverifiable state	one critical event or repeated threshold breach
Retest	component, policy, tool or task-distribution change	affected action path changed materially
Reapprove	new authority, use context or consequence	deployment moves beyond approved envelope
Retire	unresolved risk, obsolete purpose, unsupported dependency	evidence cannot justify continued operation

Every signal needs an owner, threshold, response time and evidence-retention rule. A red indicator with no decision right is theatre. An alert that fires on every minor variation becomes background noise.

Automate evidence that has a stable meaning

Good candidates for automation include:

• version and configuration attestation;
• tool, permission and network inventories;
• schema, policy and deterministic business-rule checks;
• replay of regression and adversarial test suites;
• sampling and scoring of privacy-safe trajectories;
• checks for duplicate, missing or prohibited actions;
• comparison of expected and authoritative end state;
• drift in input, route, latency, cost and failure categories; and
• control health, revocation and rollback exercises.

Automation should preserve provenance: which system version produced the evidence, under which data and test version, using which evaluator. An unexplained score is not an assurance artefact.

The public discussion of AI assurance is converging on a lifecycle view. For example, Ofgem’s 2026 call for input describes assurance as processes, measures and evidence used to assess, monitor, test, evaluate and communicate whether systems meet intended objectives, obligations and expectations. It lists technical testing, monitoring, documentation, human oversight and independent review among possible activities. This remains an evolving policy area, not a universal certification scheme.

Keep human judgement where meaning is contested

Automated systems can check a declared rule. People must decide whether the rule captures the concern. Human judgement remains necessary for:

• defining acceptable outcomes and risk tolerance;
• deciding whether a failure is material;
• evaluating impacts that are hard to quantify;
• resolving conflicts between quality, fairness, privacy, safety and cost;
• reviewing complaints, appeals and affected-party evidence;
• accepting residual risk or changing authority; and
• deciding when independent review is required.

“Human in the loop” is not enough. Name a role with time, competence, authority and access to the underlying evidence. Separate system owners who benefit from release from reviewers where consequence or conflict warrants it.

The UK NCSC’s agentic AI guidance stresses ongoing visibility and meaningful human oversight and control. Oversight becomes meaningful when it can pause, narrow or reverse operation—not when it merely receives a monthly report.

Treat automated evaluators as governed components

Model-based judges can review natural-language quality or classify traces at scale. They can also be inconsistent, biased towards particular styles, sensitive to prompts and blind to the same errors as the system being judged. Their verdicts are evidence with uncertainty, not ground truth.

For each automated evaluator:

• define the criterion in observable terms;
• calibrate against domain reviewers and objective checks;
• measure false accept and false reject rates on relevant cases;
• protect evaluation prompts and holdouts from the agent under test;
• version the judge model, rubric and thresholds;
• route ambiguous or high-consequence cases to people; and
• monitor whether evaluator performance drifts.

An academic survey of LLM-agent evaluation highlights unresolved gaps in cost-efficiency, safety, robustness and fine-grained scalable methods. Automation helps scale a known measurement; it does not remove those epistemic limits.

Use layered evidence. Deterministic checks verify permissions and final state. Statistical tests monitor population changes. Model-based review triages complex traces. Domain specialists judge context and impact. Users supply complaints and appeal evidence. No one layer should be asked to establish everything.

Preserve challenge and independence

Continuous operation can normalise gradual degradation. Teams adapt thresholds to avoid alerts, exceptions become routine and successful volume obscures rare harm. Schedule reviews that question the measurement system itself:

• Are the metrics still connected to the intended outcome?
• Are important groups or task variants missing?
• Do production samples exclude the hardest failures?
• Has the authority envelope widened through configuration?
• Are evaluators learning the same blind spots as the agent?
• Did incident and complaint evidence change the risk model?

Independent review can add value for consequential decisions, but independence should be described precisely: organisational separation, freedom from delivery incentives, access to evidence and authority to report findings. It should not be implied by an automated product badge.

Operate two cadences and event triggers

Use a fast operational loop for control health, serious anomalies and immediate response. Use a slower governance loop for cumulative performance, residual risk, changes in use, complaints, ownership and whether the assurance method still works.

Add event-triggered review for model or tool changes, new permissions, new user groups, new jurisdictions or sectors, material incidents, altered business processes and demonstrated new capabilities. A release calendar alone cannot anticipate those events.

Keep the assurance record proportionate: current system manifest, approved operating envelope, test evidence, material production signals, incidents, decisions, owners and open actions. Retain enough trajectory evidence to investigate while applying privacy, security and retention limits.

Automate observation; keep accountability explicit

The near-term goal is not an autonomous auditor that watches an autonomous agent and approves its own remedies. That arrangement compounds correlated errors and obscures who accepted risk.

A stronger system automates repeatable observation and controlled tests, exposes uncertainty, and brings material decisions to accountable people. It can narrow authority automatically under pre-approved stop rules, but restoration should require evidence and explicit authorisation when consequences are significant.

Continuous assurance is valuable because agentic systems keep changing. It remains assurance only when evidence leads to challenge, action and a named decision—not when automation converts uncertainty into a green light.