Failure-propagation analysis
Decide how failures spread and where containment or recovery should occur
When one component failure creates unexpected downstream behaviour, this area becomes relevant. For the platform reliability and system architecture owner, the concern is specific: a recoverable retrieval failure becomes an incorrect tool action downstream. The deployment evaluation starts from that consequence and follows the relevant evidence through the customer workflow. Follow the failure through component boundaries and record where uncertainty is lost. The most consequential issue may be a downstream interpretation that turns an explicit error into an apparently valid business fact.
Connect the operating question to inspectable evidence
- Platform reliability and system architecture owner
Owns the bounded decision and the consequence of acting.
- Component dependency map
Defines the starting context for the proposed work.
- Error translation
Exposes the first material uncertainty to examine.
- Containment and recovery priorities
Carries unresolved conditions into an accountable next action.
Decision at stake
Decide how failures spread and where containment or recovery should occur
Make the decision explicit
- Platform reliability and system architecture owner
- One component failure creates unexpected downstream behaviour.
- Decision
- How failures spread and where containment or recovery should occur.
- Error translation
- Use component dependency map to establish the relevant starting condition.
For the platform reliability and system architecture owner, this becomes relevant when one component failure creates unexpected downstream behaviour. The immediate decision is how failures spread and where containment or recovery should occur. The scope should be narrow enough that the evaluation team can identify a meaningful alternative, including continuing the current approach if the available evidence already supports it.
The practical failure to challenge is that a recoverable retrieval failure becomes an incorrect tool action downstream. That concern gives the work a concrete direction: establish what the relevant boundary should do, inspect how it behaves or would operate, and identify the owner of any gap. It should not be diluted into a broad transformation exercise before the specific question has been answered.
Coverage
Examine error translation alongside state contamination
Keep the review distinctions visible
- Error translation
- Follow the meaning of a fault as it crosses component interfaces.
- State contamination
- Check whether a failed component leaves trusted but invalid shared state.
- Cascading retries or authority changes
- Observe whether the fault triggers cascading retries or broader permissions.
Error translation
For this scope, error translation should be examined alongside component dependency map. Track how each component represents the injected fault. An error may become an empty result, a missing field, or an apparently valid fact. The proposed analysis should identify the exact translation that changes its meaning and whether downstream components still know the task evidence is incomplete.
State contamination
For this scope, state contamination should be examined alongside fault injection boundary. Inspect state written after the failure, including plans, summaries, and memory where relevant. Incorrect intermediate state can affect later tasks even if the immediate run stops. The evaluation should preserve the path from fault to persistent influence and identify the owner of the affected representation.
Cascading retries or authority changes
For this scope, cascading retries or authority changes should be examined alongside expected degraded behaviour. Observe retries and secondary actions triggered by the fault. A local recovery attempt may amplify load or create unintended effects elsewhere. The finding should distinguish the initiating problem from the propagation mechanism and recommend a containment check that can be retested on the same controlled trajectory.
Prerequisites
Start with component dependency map
Give each input a specific job
- Component dependency map
- Defines the operating boundary.
- Fault injection boundary
- Examine state contamination.
- Expected degraded behaviour
- Shows conditions affecting cascading retries or authority changes.
- Product or platform owner
- Confirm access and a reviewer who can resolve gaps.
The starting material is component dependency map, fault injection boundary, and expected degraded behaviour. Review these together rather than assuming one document tells the whole story. A stated rule may differ from a configured behaviour, and a proposed responsibility may not yet have an available owner. Identify those differences before selecting the detailed work.
Participation should include a product or platform decision owner, a technical lead, a data owner, and a competent reviewer for the relevant boundary. The customer should define which information and environments may be used. Synthetic or sanitised examples can support preparation when they preserve the relevant behaviour; if they omit a material dependency, record the limitation and agree how the missing evidence would be obtained.
How the work proceeds
Tie the deployment finding to observable behaviour
Follow the evidence through the work
- Error translation
Use component dependency map to define the decision criteria.
- State contamination
Inspect fault injection boundary and retain observations that challenge the expected result.
- Cascading retries or authority changes
Use expected degraded behaviour to identify unresolved conditions and a responsible owner.
- Containment and recovery priorities
Link the recommendation to its evidence, remaining limits and next validation condition.
Inject one controlled fault at a time where possible and trace its representations downstream. Observe whether components preserve error status, uncertainty, and affected state or translate them into apparently valid facts. The proposed analysis should distinguish direct effects from cascading retries, secondary tool actions, and misleading completion messages.
A practical first slice would focus on error translation and then follow its consequences into state contamination. Keep cascading retries or authority changes visible as a separate review question rather than assuming it is satisfied by the first result. The evaluation team should record observations, competing explanations, and changes to the original assumptions. A second reviewer can challenge whether the evidence supports the recommendation and whether an apparently successful example omitted the condition that matters most.
Illustrative scenario
Missing field becomes eligibility: contain the conversion
Examine the constructed decision boundary
Illustrative scenario
- Missing field becomes eligibility
Incomplete tool data reaches a planner that treats absence as permission.
- Fault-to-action trace
Remove the eligibility field and follow where uncertainty becomes an affirmative decision.
- Contain the conversion
Repair that boundary and verify unrelated valid work remains possible.
Consider an illustrative situation in which a tool returns incomplete account data and the planner proceeds as if the missing field confirmed eligibility. This is a constructed example, not a client engagement or an observed Softlander outcome.
For incomplete account data, an illustrative run would remove a required eligibility field and follow the planner’s response. The failure may originate in the tool but become consequential only when absence is interpreted as approval. The remediation should target that conversion and verify that containment does not unnecessarily block unrelated valid work.
The useful learning would be the difference between the expected behaviour and the behaviour or responsibility actually demonstrated. If the exercise leaves cascading retries or authority changes unresolved, that should remain a named next step. A recommendation to narrow the scope, improve a prerequisite, or hold the decision can be appropriate when the example exposes a material dependency.
Indicative artefacts
Use containment and recovery priorities to make the next action explicit
Every artefact needs an operating use
Illustrative working format
- Failure propagation map
- Boundary — component dependency map.
- Controlled fault findings
- Findings about state contamination.
- Containment and recovery priorities
- Action on cascading retries or authority changes.
Indicative outputs include failure propagation map, controlled fault findings, and containment and recovery priorities. Their form should follow the decision and the customer’s existing operating records. The first should establish the assessed boundary and relevant observations; the second should make material gaps or dependencies inspectable; the third should identify the action, owner, and evidence needed to revisit the decision.
Acceptance should be tested through use: can the responsible owner explain what was examined, identify an unresolved issue, and act on the next step without reconstructing the whole engagement? Findings should preserve counterevidence and limits. A polished document is insufficient if it does not help resolve the customer’s specific question.
Scope and authority
A deployment finding remains bounded to the tested conditions
Keep analytical work separate from decision authority
- Authorised deployment cases and traces
- Can include failure propagation map.
- Product or platform owner
- The customer retains release authority and owns remediation priorities.
- Universal vendor or component assurance
- Cannot be inferred from controlled fault findings.
The proposed evaluation concerns the named deployment, configuration, cases, and authorised environment. Findings do not certify a component vendor, establish universal safety, or approve release. Unexamined provider paths and specialist questions remain explicit limitations.
Ways to engage
Related engagement scopes
AI Agent Evaluation Evidence Diagnostic
Find the gaps between your release decision and the evidence behind it. Start with one agent workflow and a bounded set of questions.
AI Agent Evaluation Foundation
Turn scattered tests into a repeatable release discipline. Connect the behaviours that matter to datasets, evaluation criteria and evidence your team can maintain.
AI Agent Component Evaluation
Find where agent behaviour breaks down. Follow memory, retrieval and delegation through your workflow, then isolate the components that need attention.
Continue exploring
Related insights and resources
Engineer the harness, not just the prompt
Define the contracts around an agent's decisions: what state it trusts, which actions can execute, how interrupted work resumes, and what evidence an operator needs to recover safely.
What would justify releasing this agent?
Replace a single pass rate with a reviewable chain of requirements, cases, observations and judgements. Distinguish task quality, control failures and uncertainty before increasing an agent's exposure.
The evidence behind the go-live decision
A working paper for leaders who need to explain why an agent workflow is ready for its next release. Includes a proposed evidence-pack structure, decision table and review agenda.