Termination-condition testing
Determine whether termination conditions match actual business completion
When agents stop too early or continue after the task is complete, this area becomes relevant. For the workflow release and operations owner, the concern is specific: the agent declares completion after requesting an action that has not actually succeeded. The deployment evaluation starts from that consequence and follows the relevant evidence through the customer workflow. Termination should be tied to verified outcomes and explicit limits. Tests should distinguish completed work, partial completion, a blocked task, user cancellation, and a resource limit so each state produces an accurate explanation.
Connect the operating question to inspectable evidence
- Workflow release and operations owner
Owns the bounded decision and the consequence of acting.
- Completion criteria
Defines the starting context for the proposed work.
- Premature success
Exposes the first material uncertainty to examine.
- Stopping and escalation rules
Carries unresolved conditions into an accountable next action.
Decision at stake
Determine whether termination conditions match actual business completion
Make the decision explicit
- Workflow release and operations owner
- Agents stop too early or continue after the task is complete.
- Decision
- Whether termination conditions match actual business completion.
- Premature success
- Use completion criteria to establish the relevant starting condition.
For the workflow release and operations owner, this becomes relevant when agents stop too early or continue after the task is complete. The immediate decision is whether termination conditions match actual business completion. The scope should be narrow enough that the evaluation team can identify a meaningful alternative, including continuing the current approach if the available evidence already supports it.
The practical failure to challenge is that the agent declares completion after requesting an action that has not actually succeeded. That concern gives the work a concrete direction: establish what the relevant boundary should do, inspect how it behaves or would operate, and identify the owner of any gap. It should not be diluted into a broad transformation exercise before the specific question has been answered.
Coverage
Examine premature success alongside unresolved effects
Keep the review distinctions visible
- Premature success
- Challenge a success message before the business completion condition holds.
- Unresolved effects
- Inspect pending and uncertain effects before accepting termination.
- Cancellation and limit handling
- Test cancellation and exhausted budgets with unfinished work still visible.
Premature success
For this scope, premature success should be examined alongside completion criteria. Create a case where the agent can produce a convincing answer before the business task is complete. Require verification of any external effect that defines completion. The proposed evaluation should reveal whether the stopping rule follows the actual task contract or only the agent’s sense that its response is finished.
Unresolved effects
For this scope, unresolved effects should be examined alongside task state traces. Interrupt a workflow after some effects occur and inspect the final status. Partial completion should retain unresolved steps and relevant evidence. The test should distinguish a blocked task from success and should examine whether the user receives enough information to decide on continuation or recovery.
Cancellation and limit handling
For this scope, cancellation and limit handling should be examined alongside budget and cancellation rules. Cancel work or trigger an agreed limit while an external event remains pending. Inspect whether late events restart actions or change the reported outcome. The termination contract should explain which work remains authorised after cancellation and who must resolve any effect already in progress.
Prerequisites
Start with completion criteria
Give each input a specific job
- Completion criteria
- Defines the operating boundary.
- Task state traces
- Examine unresolved effects.
- Budget and cancellation rules
- Shows conditions affecting cancellation and limit handling.
- Product or platform owner
- Confirm access and a reviewer who can resolve gaps.
The starting material is completion criteria, task state traces, and budget and cancellation rules. Review these together rather than assuming one document tells the whole story. A stated rule may differ from a configured behaviour, and a proposed responsibility may not yet have an available owner. Identify those differences before selecting the detailed work.
Participation should include a product or platform decision owner, a technical lead, a data owner, and a competent reviewer for the relevant boundary. The customer should define which information and environments may be used. Synthetic or sanitised examples can support preparation when they preserve the relevant behaviour; if they omit a material dependency, record the limitation and agree how the missing evidence would be obtained.
How the work proceeds
Tie the deployment finding to observable behaviour
Follow the evidence through the work
- Premature success
Use completion criteria to define the decision criteria.
- Unresolved effects
Inspect task state traces and retain observations that challenge the expected result.
- Cancellation and limit handling
Use budget and cancellation rules to identify unresolved conditions and a responsible owner.
- Stopping and escalation rules
Link the recommendation to its evidence, remaining limits and next validation condition.
Create distinct cases for completed, blocked, partially completed, cancelled, and resource-limited tasks. Require evidence of external effects where completion depends on them. The proposed test should also examine whether a late event can restart work after cancellation or whether a premature success message conceals unresolved obligations.
A practical first slice would focus on premature success and then follow its consequences into unresolved effects. Keep cancellation and limit handling visible as a separate review question rather than assuming it is satisfied by the first result. The evaluation team should record observations, competing explanations, and changes to the original assumptions. A second reviewer can challenge whether the evidence supports the recommendation and whether an apparently successful example omitted the condition that matters most.
Illustrative scenario
Success before tool completion: completion-rule repair
Examine the constructed decision boundary
Illustrative scenario
- Success before tool completion
An agent reports a case update before the connector returns its final result.
- Delayed rejection
Leave the operation pending, then return a failure.
- Completion-rule repair
Require accurate pending and failed states; adjust event handling or result interpretation.
Consider an illustrative situation in which a case agent sends an update request and reports success before the connector returns its final result. This is a constructed example, not a client engagement or an observed Softlander outcome.
For an update request awaiting a final result, an illustrative case would delay and then reject the operation. The agent should preserve the pending state and explain the final failure accurately. The owner can then decide whether completion criteria, event handling, or tool-result interpretation needs adjustment.
The useful learning would be the difference between the expected behaviour and the behaviour or responsibility actually demonstrated. If the exercise leaves cancellation and limit handling unresolved, that should remain a named next step. A recommendation to narrow the scope, improve a prerequisite, or hold the decision can be appropriate when the example exposes a material dependency.
Indicative artefacts
Use stopping and escalation rules to make the next action explicit
Every artefact needs an operating use
Illustrative working format
- Termination case set
- Boundary — completion criteria.
- Completion-state findings
- Findings about unresolved effects.
- Stopping and escalation rules
- Action on cancellation and limit handling.
Indicative outputs include termination case set, completion-state findings, and stopping and escalation rules. Their form should follow the decision and the customer’s existing operating records. The first should establish the assessed boundary and relevant observations; the second should make material gaps or dependencies inspectable; the third should identify the action, owner, and evidence needed to revisit the decision.
Acceptance should be tested through use: can the responsible owner explain what was examined, identify an unresolved issue, and act on the next step without reconstructing the whole engagement? Findings should preserve counterevidence and limits. A polished document is insufficient if it does not help resolve the customer’s specific question.
Scope and authority
A deployment finding remains bounded to the tested conditions
Keep analytical work separate from decision authority
- Authorised deployment cases and traces
- Can include termination case set.
- Product or platform owner
- The customer retains release authority and owns remediation priorities.
- Universal vendor or component assurance
- Cannot be inferred from completion-state findings.
The proposed evaluation concerns the named deployment, configuration, cases, and authorised environment. Findings do not certify a component vendor, establish universal safety, or approve release. Unexamined provider paths and specialist questions remain explicit limitations.
Ways to engage
Related engagement scopes
AI Agent Evaluation Evidence Diagnostic
Find the gaps between your release decision and the evidence behind it. Start with one agent workflow and a bounded set of questions.
AI Agent Evaluation Foundation
Turn scattered tests into a repeatable release discipline. Connect the behaviours that matter to datasets, evaluation criteria and evidence your team can maintain.
AI Agent Component Evaluation
Find where agent behaviour breaks down. Follow memory, retrieval and delegation through your workflow, then isolate the components that need attention.
Continue exploring
Related insights and resources
Engineer the harness, not just the prompt
Define the contracts around an agent's decisions: what state it trusts, which actions can execute, how interrupted work resumes, and what evidence an operator needs to recover safely.
What would justify releasing this agent?
Replace a single pass rate with a reviewable chain of requirements, cases, observations and judgements. Distinguish task quality, control failures and uncertainty before increasing an agent's exposure.
The evidence behind the go-live decision
A working paper for leaders who need to explain why an agent workflow is ready for its next release. Includes a proposed evidence-pack structure, decision table and review agenda.