Snapshot and rollback testing
Determine what a snapshot can restore after an agent failure
The recovery plan assumes restoring a worker reverses all effects. The decision is how to determine what a snapshot can restore after an agent failure. The work begins with your actual workflow and looks for the evidence that separates intended behaviour from a consequential exception. It is particularly relevant to SRE and stateful workflow owners. A useful outcome is an explicit account of what was examined, what remains uncertain, and which decision the evidence can support.
The runtime isolation relationship
- Snapshot scope
Establish the basis for snapshot contents.
- External effects
Identify writes beyond the restore boundary.
- Rollback boundary map
Retain the scoped finding and its unresolved dependencies.
Decision
Frame the decision around snapshot contents
Questions and the evidence they need
- Snapshot contents
- Inspect snapshot scope for the relevant constraint or observation.
- External effects
- Inspect external tool state for the relevant constraint or observation.
- Reconciliation
- Inspect storage dependencies for the relevant constraint or observation.
The decision is how to determine what a snapshot can restore after an agent failure. This is relevant to SRE and stateful workflow owners when the recovery plan assumes restoring a worker reverses all effects. Name the affected workflow, the consequence of a wrong decision, and the owner who can act on the result. The initial discussion should establish which evidence is already available and which uncertainty is material enough to investigate.
The specific failure hypothesis is that a restored sandbox replays an action whose external side effect already happened. It remains a hypothesis until examined in the deployment. Existing internal tests or operating records may already resolve it; identify the uncovered question before commissioning additional work.
Mechanism
A restored snapshot may not restore external business state
Relationships within runtime isolation
- Snapshot contents
- Name captured runtime and storage state.
- External effects
- Identify writes beyond the restore boundary.
- Reconciliation
- Define how restored and live states are compared.
What this specialty examines
Snapshots restore a defined set of state, not every consequence of an agent’s activity. Identify included files and memory, external systems, credentials and pending jobs. Specify consistency assumptions and what must be reconciled after restoration.
The review should exercise snapshot creation, restore, consistency checks and external-state reconciliation. Identify the normal route, relevant exceptions and the authoritative record of the outcome. Inspect the relevant business-system record, enforcement event or process evidence alongside any reported status. Keep incomplete cases visible, with an explanation of what their absence prevents the owner from concluding.
The tradeoff to make explicit
For this decision, snapshots speed local recovery while external actions may need separate compensation. Compare the available design or operating choices against the same intended outcome. If one option removes a required control or changes the task boundary, describe that difference before treating its apparent simplicity, speed or lower effort as an improvement. The decision should expose the tradeoff instead of burying it in an aggregate score.
Preparation
Connect the working inputs to the boundary being examined
Evidence packet for snapshot contents
Illustrative working format
- Snapshot scope
- Identify retained state and distinguish repeated work from repeated external effects.
- External tool state
- Check its relationship to the relevant workflow and expose missing context.
- Storage dependencies
- Name the custodian and the conditions requiring a refreshed version.
- Recovery procedures
- Identify provenance and the specific decision this material can inform.
Start with snapshot scope, external tool state, storage dependencies and recovery procedures. Identify the owner, revision and operating context of each source. Sanitised examples may support initial design, but differences from actual conditions should remain explicit. Where direct access is unsuitable, a customer-controlled evidence or execution route may be considered, with agreed handling and retention.
The scope also needs the runtime owner, agreed expectations for a representative boundary test, and the authority to inspect the relevant boundary. Specify what evidence would distinguish the failure in which a restored sandbox replays an action whose external side effect already happened from acceptable behaviour. If stakeholders disagree about intended behaviour, resolve the interpretation before classifying the observed difference as a defect.
How the work proceeds
Carry snapshot contents through to an accountable finding
From the scoped material to a decision record
- Snapshot scope
Declare the permitted environment and resources.
- External effects
Apply the scoped check to reconciliation.
- Rollback boundary map
Record the evidence, open dependencies, and the owner’s next decision.
Prepare the runtime boundary and fixtures
Use snapshot scope, external tool state, storage dependencies and recovery procedures to identify the exact runtime and boundary claimed by SRE and stateful workflow owners. Mark permitted resources and protected resources in a controlled fixture, including shared services where they are relevant. Define observable success, denial and incomplete setup before execution. The preparation output is a boundary map and fixture manifest that lets reviewers distinguish effective isolation from a test that never reached its target.
Exercise and inspect the boundary
Test restoration against a known checkpoint and inspect both local and external state. A replayed worker may repeat actions that already committed elsewhere. Recovery needs a way to recognise those effects or transfer reconciliation to an accountable operator.
Choose containment or configuration changes
Translate the observations into a rollback boundary map and restoration test record. Locate each issue in the worker, shared service, policy rollout or cleanup path rather than attributing every symptom to the sandbox as a whole. Decide which configuration change, narrower resource exposure or specialist investigation is warranted. Retest the affected boundary and inspect residual state before treating the changed environment as equivalent to the intended design.
Indicative artefact
Make the rollback boundary map usable by its owner
Rollback boundary map
Illustrative working format
- Rollback boundary map
- Identify the assessed workflow, revision, and decision boundary.
- Restoration test record
- Link supporting and contrary observations to inspectable evidence.
- External effects evidence
- Link observations and counterevidence to the case, configuration, and conclusion limit.
A useful decision artefact is a rollback boundary map and restoration test record. Supporting material may include selected cases, configuration or process records, evidence references, a disagreement log and prioritised actions. Retain the case selection rationale and material disagreements alongside the result; otherwise the result may be mistaken for complete coverage.
Illustrative scenario
Examine reconciliation through a constructed exception
A constructed case and its decision paths
Illustrative scenario
- Snapshot contents
A restored worker repeats an external operation completed before the snapshot rollback.
- External effects
Compare restored local state with the external operation record.
- Reconciliation
Reconcile or compensate the external effect before replaying work.
Consider a fictional situation: a code agent updates an external issue tracker, then its filesystem snapshot is restored to an earlier point. The scenario is illustrative, with no client engagement, executed test or measured improvement implied.
For the issue-tracker update, preserve the external action identifier before restoring local files. The workflow should inspect existing state before repeating the update. Snapshot success and business recovery success are separate outcomes to report.
Scope and next action
Keep the conclusion within the authorised runtime boundary
Confirm fit before the next commitment
- Snapshot contents
- This review is limited to the authorised runtime boundary.
- Snapshot scope
- The relevant owner must confirm access and the intended operating conditions.
- Rollback boundary map
- Untested surfaces remain outside the conclusion.
Testing is confined to explicitly authorised environments and non-destructive cases. A bounded assessment cannot establish that escape or cross-boundary access is impossible. Any engagement depends on an agreed scope, qualified team, authorised access and confirmed availability. Bring the affected workflow, its current configuration or process, the triggering change and the evidence already available. The immediate scope question is prompted by the situation in which the recovery plan assumes restoring a worker reverses all effects; confirm which missing observation would change the owner’s next action.
Ways to engage
Related engagement scopes
AI Agent Runtime Security Review
Make every delegation explicit. Examine who the agent acts for, where it can execute and which tools it can reach when instructions alone are not enough.
AI Agent Red Team
Challenge the boundaries that matter to your workflow. Test agreed attack scenarios against agent instructions, tools and data access.
AI Agent Sandbox Assurance
Examine the boundary between agent execution and the systems around it, including files, networks, secrets and cleanup.
Continue exploring
Related insights and resources
The permission boundary behind every agent action
An agent can only be evaluated in the context of what its tools allow. Examine identity, delegated authority and action boundaries before you put a workflow in motion.
Before your agent can act, define its authority
A working paper for the moment an AI system moves from advice to action. Map identity, permissions, approvals and recovery into an authority model your team can review and test.
Computer-use agents bring the interface back into the threat model
A computer-use agent treats pixels, accessibility data and interface state as both evidence and an action surface. Security must bind intent to effects beyond the screen.