Continuous evaluation without alert theatre

A dashboard can glow red all day while nobody makes a different decision. That is alert theatre: activity that signals concern without creating a reliable path from evidence to action.

Continuous evaluation is valuable for agent systems because prompts, models, tools, data and user behaviour change. But running checks continuously does not mean paging continuously. The operating design must separate what is measured, what is reviewed, what interrupts a person and what automatically constrains the system.

The goal is not more alerts. It is shorter time from meaningful loss of control to an appropriate response.

Start with operating decisions

List the decisions continuous evidence must support:

• keep the agent operating within current authority;
• reduce scope or require more human review;
• roll back a recent configuration;
• disable a tool or credential;
• investigate a possible incident;
• refresh an evaluation set or threshold;
• approve a planned change.

For each decision, name an owner, time horizon and evidence. A security operator may need an immediate signal for unauthorised outbound action. A product owner may review unnecessary handoffs weekly. An evaluation lead may inspect grader drift monthly.

The NIST AI RMF calls for post-deployment monitoring, incident response, recovery and change management, as well as ongoing tracking of emerging risk. It does not imply that every measurement should become an interruptive alert. Governance defines which signals drive which treatments.

Separate four evidence lanes

Use different handling for different time horizons:

Lane	Purpose	Typical response
Control event	Detect a hard boundary failure	Deny, contain and page if consequence is urgent
Service signal	Detect user-visible degradation	Page or roll back based on agreed objective
Evaluation result	Detect behavioural regression	Open a review, constrain release or schedule remediation
Learning signal	Discover emerging patterns	Analyse trends and update cases or policy

A failed nightly adversarial case is important, but it may not justify waking an operator if the affected capability is disabled. A single confirmed unauthorised production action may justify immediate containment even before an aggregate threshold is crossed.

This separation reduces the temptation to put every concern on one dashboard with one red-green status.

Alert on consequence and failed control

Page when a person can take a specific, time-sensitive action. Good candidates include:

• a protected action executed without valid approval;
• sensitive data sent to an unauthorised destination;
• repeated writes or irreconcilable transaction state;
• an emergency stop or policy service that failed open;
• a sharp loss of qualified completion after a release;
• a queue or latency failure that breaks a user commitment;
• inability to observe a control whose evidence is required for operation.

Prefer user or workflow symptoms over internal curiosities. Google’s SRE guidance on monitoring distributed systems distinguishes symptoms from causes and warns that frequent pages lead people to skim or ignore alerts. Agent monitoring should use internal traces for diagnosis while paging on a clear loss of service or control.

Model confidence, token count or prompt sentiment can be useful exploratory signals. They rarely justify an interrupt on their own unless validated against a consequence and paired with a response.

Design thresholds with volume and uncertainty

A fixed percentage can mislead when traffic is sparse. One failure in two cases looks catastrophic; ten failures in ten thousand may still be unacceptable if they cross an authority boundary. Use different threshold forms:

• zero-event gate for confirmed critical control bypass;
• rate plus minimum volume for common quality regressions;
• rolling window for sustained deterioration;
• change from baseline for configuration regressions;
• tail threshold for latency, cost or step explosion;
• case-specific trigger for a known high-consequence incident pattern.

Show denominator, window and uncertainty. Avoid alerting on a model-graded signal unless grader reliability and disagreement are monitored too.

Continuous monitoring has a longer history in security. NIST SP 800-137 describes monitoring as visibility into assets, threats, vulnerabilities and control effectiveness so organisations can respond in line with risk tolerance. The transferable principle is alignment: collect and escalate what informs timely risk response.

Measure every alert as a control

An alert is not successful because it fired. Track:

• precision: how many alerts represented a real actionable condition;
• coverage: which consequential conditions have detection;
• time to detect and acknowledge;
• time to contain or reach stable disposition;
• repeated alerts for the same unresolved cause;
• alerts closed without action and why;
• operator feedback and unsafe workarounds;
• whether the stated response actually worked.

The healthcare literature provides a sobering analogy. The AHRQ Patient Safety Network primer on alert fatigue explains how large volumes of inconsequential warnings can desensitize clinicians to meaningful ones. Agent operations are not clinical care, but the human-factors lesson transfers: specificity, severity tiers and interrupting only for consequential conditions protect attention.

Review noisy alerts as defects. Tune, downgrade, aggregate or remove them. An alert with no owner or runbook should not page.

Build a three-level response model

Use clear levels tied to action:

Immediate containment. For confirmed severe boundary loss, automatically deny or disable the affected path where safe, preserve evidence and notify the on-call owner.

Time-bounded review. For meaningful regression without active severe consequence, create an owned item with evidence, deadline and possible scope change. Do not let it vanish into an unread channel.

Trend and learn. For emerging patterns, aggregate by workflow, configuration and case slice. Review at a defined cadence and use the result to update datasets, prompts, tools or policy.

Each alert should include current configuration, affected users or workflow, observed versus expected state, likely release change, immediate safe action and trace link. The operator should not need to reconstruct the basic event while consequence continues.

Connect change evaluation to production evidence

Run pre-release regressions when prompts, models, tools, policies, retrieval or memory change. After release, compare production behaviour with the evaluated conditions. Preserve the link from version to results.

Use three complementary mechanisms:

1. scheduled synthetic cases against representative dependencies;
2. privacy-controlled sampling of real traces against qualified outcome and policy graders;
3. incident and override intake that creates new regression candidates.

The NCSC secure AI development guidance includes logging and monitoring, update management, information sharing and incident processes in secure operation. Keep logs proportionate and access-controlled; agent traces may contain sensitive prompts, data and credentials.

Test the monitoring system itself

Simulate the conditions you expect to detect: policy denial, duplicate write, missing trace, grader outage, model drift and emergency shutdown. Confirm routing, permissions, escalation and recovery. Check that containment does not create a worse state.

Run tabletop exercises for uncertain cases. Who decides whether to disable the tool? Who can revoke credentials? How are affected users informed? Which evidence must be preserved? How does the service return safely?

Then test silent failure. If the evaluation pipeline stops running, the absence of red results must not look like good performance. Monitor freshness and completeness of the evidence itself.

Keep the operating promise small and explicit

A sustainable continuous-evaluation program may begin with five signals, not fifty: qualified completion, unauthorized-action attempts and outcomes, recovery failures, human override, and evidence-pipeline health. Add signals only when they support a named decision.

Review the catalogue regularly. Remove measurements nobody uses. Recalibrate thresholds after volume or workflow changes. Record accepted blind spots and compensating controls.

Continuous evaluation earns trust when it changes behaviour: a release is constrained, a tool is contained, a weak case becomes a regression, or a safe system continues without needless interruption. Everything else is telemetry. Useful telemetry can wait for a review; an alert must deserve a person’s attention.