Loop and deadlock detection
Determine whether loops and deadlocks are detected and resolved within agreed bounds
When workflows repeat steps or wait indefinitely, this area becomes relevant. For the agent reliability and operations owner, the concern is specific: an agent alternates between tools without gaining evidence or completing the task. The deployment evaluation starts from that consequence and follows the relevant evidence through the customer workflow. Define progress in business terms before setting iteration limits. Repetition may be appropriate when new evidence arrives; the evaluation should distinguish useful retries from cycles that consume resources without changing the decision state.
Connect the operating question to inspectable evidence
- Agent reliability and operations owner
Owns the bounded decision and the consequence of acting.
- Workflow and dependency graph
Defines the starting context for the proposed work.
- Productive versus unproductive repetition
Exposes the first material uncertainty to examine.
- Timeout and escalation recommendations
Carries unresolved conditions into an accountable next action.
Decision at stake
Determine whether loops and deadlocks are detected and resolved within agreed bounds
Make the decision explicit
- Agent reliability and operations owner
- Workflows repeat steps or wait indefinitely.
- Decision
- Whether loops and deadlocks are detected and resolved within agreed bounds.
- Productive
- Use workflow and dependency graph to establish the relevant starting condition.
For the agent reliability and operations owner, this becomes relevant when workflows repeat steps or wait indefinitely. The immediate decision is whether loops and deadlocks are detected and resolved within agreed bounds. The scope should be narrow enough that the evaluation team can identify a meaningful alternative, including continuing the current approach if the available evidence already supports it.
The practical failure to challenge is that an agent alternates between tools without gaining evidence or completing the task. That concern gives the work a concrete direction: establish what the relevant boundary should do, inspect how it behaves or would operate, and identify the owner of any gap. It should not be diluted into a broad transformation exercise before the specific question has been answered.
Coverage
Productive and unproductive repetition need different evidence
Keep the review distinctions visible
- Productive versus unproductive repetition
- Use a progress condition to distinguish useful retry from a repeated state.
- Circular waits
- Expose dependencies that wait on each other without a possible next move.
- Escalation on stalled progress
- Verify timeout and escalation when the agreed progress signal is absent.
Productive versus unproductive repetition
For this scope, productive versus unproductive repetition should be examined alongside workflow and dependency graph. Define progress as a meaningful change in task evidence or state. Include a repeated tool call that obtains new information and another that produces the same unresolved result. The evaluation should distinguish justified persistence from an unproductive cycle instead of treating every repeated step as a loop defect.
Circular waits
For this scope, circular waits should be examined alongside repeated-state traces. Construct a controlled circular dependency in which each participant waits for another. Preserve the expected acknowledgement and timeout behaviour. The proposed test should inspect whether the system identifies the wait relationship and assigns a resolution path rather than merely exhausting a resource limit.
Escalation on stalled progress
For this scope, escalation on stalled progress should be examined alongside progress and timeout definitions. Observe the status and escalation produced when progress stops. Containment, diagnosis, and recovery are separate outcomes. The owner should see whether the workflow can explain what is blocked and what decision or external event is required, rather than receive a generic failure after prolonged activity.
Prerequisites
Start with workflow and dependency graph
Give each input a specific job
- Workflow and dependency graph
- Defines the operating boundary.
- Repeated-state traces
- Examine circular waits.
- Progress and timeout definitions
- Shows conditions affecting escalation on stalled progress.
- Product or platform owner
- Confirm access and a reviewer who can resolve gaps.
The starting material is workflow and dependency graph, repeated-state traces, and progress and timeout definitions. Review these together rather than assuming one document tells the whole story. A stated rule may differ from a configured behaviour, and a proposed responsibility may not yet have an available owner. Identify those differences before selecting the detailed work.
Participation should include a product or platform decision owner, a technical lead, a data owner, and a competent reviewer for the relevant boundary. The customer should define which information and environments may be used. Synthetic or sanitised examples can support preparation when they preserve the relevant behaviour; if they omit a material dependency, record the limitation and agree how the missing evidence would be obtained.
How the work proceeds
Tie the deployment finding to observable behaviour
Follow the evidence through the work
- Productive versus unproductive repetition
Use workflow and dependency graph to define the decision criteria.
- Circular waits
Inspect repeated-state traces and retain observations that challenge the expected result.
- Escalation on stalled progress
Use progress and timeout definitions to identify unresolved conditions and a responsible owner.
- Timeout and escalation recommendations
Link the recommendation to its evidence, remaining limits and next validation condition.
Define observable progress and permissible waiting conditions before injecting stalls. Include repeated states with new evidence as well as repetitions that add nothing, and model circular dependencies between agents or tools. The proposed test should inspect detection, resource bounds, escalation, and the accuracy of the user-visible status.
A practical first slice would focus on productive versus unproductive repetition and then follow its consequences into circular waits. Keep escalation on stalled progress visible as a separate review question rather than assuming it is satisfied by the first result. The evaluation team should record observations, competing explanations, and changes to the original assumptions. A second reviewer can challenge whether the evidence supports the recommendation and whether an apparently successful example omitted the condition that matters most.
Illustrative scenario
Mutually waiting agents: recovery beyond termination
Examine the constructed decision boundary
Illustrative scenario
- Mutually waiting agents
Each participant requires the other's acknowledgement before closing a case.
- Held acknowledgments
Withhold both and inspect whether the circular dependency is recognised.
- Recovery beyond termination
Distinguish detection and resource containment from a usable explanation and resolution path.
Consider an illustrative situation in which two agents each wait for the other’s confirmation before either can close a case. This is a constructed example, not a client engagement or an observed Softlander outcome.
For mutually waiting agents, an illustrative run would hold each required acknowledgement and observe whether the workflow identifies the circular dependency. A limit that merely kills the process may contain resource use without providing useful recovery. The finding should distinguish detection, containment, explanation, and resolution.
The useful learning would be the difference between the expected behaviour and the behaviour or responsibility actually demonstrated. If the exercise leaves escalation on stalled progress unresolved, that should remain a named next step. A recommendation to narrow the scope, improve a prerequisite, or hold the decision can be appropriate when the example exposes a material dependency.
Indicative artefacts
Use timeout and escalation recommendations to make the next action explicit
Every artefact needs an operating use
Illustrative working format
- Loop and deadlock cases
- Boundary — workflow and dependency graph.
- Progress detection findings
- Findings about circular waits.
- Timeout and escalation recommendations
- Action on escalation on stalled progress.
Indicative outputs include loop and deadlock cases, progress detection findings, and timeout and escalation recommendations. Their form should follow the decision and the customer’s existing operating records. The first should establish the assessed boundary and relevant observations; the second should make material gaps or dependencies inspectable; the third should identify the action, owner, and evidence needed to revisit the decision.
Acceptance should be tested through use: can the responsible owner explain what was examined, identify an unresolved issue, and act on the next step without reconstructing the whole engagement? Findings should preserve counterevidence and limits. A polished document is insufficient if it does not help resolve the customer’s specific question.
Scope and authority
A deployment finding remains bounded to the tested conditions
Keep analytical work separate from decision authority
- Authorised deployment cases and traces
- Can include loop and deadlock cases.
- Product or platform owner
- The customer retains release authority and owns remediation priorities.
- Universal vendor or component assurance
- Cannot be inferred from progress detection findings.
The proposed evaluation concerns the named deployment, configuration, cases, and authorised environment. Findings do not certify a component vendor, establish universal safety, or approve release. Unexamined provider paths and specialist questions remain explicit limitations.
Ways to engage
Related engagement scopes
AI Agent Evaluation Evidence Diagnostic
Find the gaps between your release decision and the evidence behind it. Start with one agent workflow and a bounded set of questions.
AI Agent Evaluation Foundation
Turn scattered tests into a repeatable release discipline. Connect the behaviours that matter to datasets, evaluation criteria and evidence your team can maintain.
AI Agent Component Evaluation
Find where agent behaviour breaks down. Follow memory, retrieval and delegation through your workflow, then isolate the components that need attention.
Continue exploring
Related insights and resources
Engineer the harness, not just the prompt
Define the contracts around an agent's decisions: what state it trusts, which actions can execute, how interrupted work resumes, and what evidence an operator needs to recover safely.
What would justify releasing this agent?
Replace a single pass rate with a reviewable chain of requirements, cases, observations and judgements. Distinguish task quality, control failures and uncertainty before increasing an agent's exposure.
The evidence behind the go-live decision
A working paper for leaders who need to explain why an agent workflow is ready for its next release. Includes a proposed evidence-pack structure, decision table and review agenda.