Post-launch optimisation partnership
Decide which post-launch changes deserve controlled experimentation
When an agent is live but its quality, cost, or adoption needs attention, this area becomes relevant. For the product and operations sponsor, the concern is specific: optimisation improves a local metric while increasing human review or downstream errors. The operating model design starts from that consequence and follows the relevant evidence through the customer workflow. Post-launch work should begin with observed operating friction. The collaboration should account for human effort, rework, and failures, and should distinguish a measurable improvement from a change that merely moves cost elsewhere.
Connect the operating question to inspectable evidence
- Product and operations sponsor
- Owns the bounded decision and the consequence of acting.
- Operating baseline
- Defines the starting context for the proposed work.
- Quality-adjusted outcomes
- Exposes the first material uncertainty to examine.
- Change review cadence
- Carries unresolved conditions into an accountable next action.
Decision at stake
Decide which post-launch changes deserve controlled experimentation
Make the decision explicit
- Product and operations sponsor
- An agent is live but its quality, cost, or adoption needs attention.
- Decision
- Which post-launch changes deserve controlled experimentation.
- Quality-adjusted outcomes
- Use operating baseline to establish the relevant starting condition.
For the product and operations sponsor, this becomes relevant when an agent is live but its quality, cost, or adoption needs attention. The immediate decision is which post-launch changes deserve controlled experimentation. The scope should be narrow enough that the customer leadership team can identify a meaningful alternative, including continuing the current approach if the available evidence already supports it.
The practical failure to challenge is that optimisation improves a local metric while increasing human review or downstream errors. That concern gives the work a concrete direction: establish what the relevant boundary should do, inspect how it behaves or would operate, and identify the owner of any gap. It should not be diluted into a broad transformation exercise before the specific question has been answered.
Coverage
Examine quality-adjusted outcomes alongside experiment isolation
Keep the review distinctions visible
- Quality-adjusted outcomes
- Count accepted outcomes after human review and correction effort.
- Experiment isolation
- Change one relevant condition while preserving a credible comparison.
- Regression and rollback readiness
- Protect established behaviour and identify how a harmful change is reversed.
Quality-adjusted outcomes
For this scope, quality-adjusted outcomes should be examined alongside operating baseline. Measure accepted completed work and its quality, not only agent activity. Include available human review, correction, and failure effort. The optimisation decision should distinguish a genuine operating improvement from a change that shifts work to another team or makes incomplete outputs look faster.
Experiment isolation
For this scope, experiment isolation should be examined alongside failure and rework evidence. Define controlled experiments with comparable tasks and explicit hypotheses. Avoid changing several components without a way to interpret the result. The proposed collaboration should preserve negative or inconclusive findings so the next optimisation choice follows observed evidence rather than preference.
Regression and rollback readiness
For this scope, regression and rollback readiness should be examined alongside change capacity. Retain regression and rollback or recovery paths for material changes. An improvement on one metric can damage another workflow boundary. The operating cadence should identify which cases must be rerun and who decides whether the observed tradeoff is acceptable for the customer deployment.
Prerequisites
Start with operating baseline
Give each input a specific job
- Operating baseline
- Defines the operating boundary.
- Failure and rework evidence
- Examine experiment isolation.
- Change capacity
- Shows conditions affecting regression and rollback readiness.
- Executive sponsor
- Confirm access and a reviewer who can resolve gaps.
The starting material is operating baseline, failure and rework evidence, and change capacity. Review these together rather than assuming one document tells the whole story. A stated rule may differ from a configured behaviour, and a proposed responsibility may not yet have an available owner. Identify those differences before selecting the detailed work.
Participation should include an executive sponsor, accountable delivery owners, and the people who can accept or implement the resulting decisions. The customer should define which information and environments may be used. Synthetic or sanitised examples can support preparation when they preserve the relevant behaviour; if they omit a material dependency, record the limitation and agree how the missing evidence would be obtained.
How the work proceeds
Make the mandate usable in operating decisions
Follow the evidence through the work
- Quality-adjusted outcomes
Use operating baseline to define the decision criteria.
- Experiment isolation
Inspect failure and rework evidence and retain observations that challenge the expected result.
- Regression and rollback readiness
Use change capacity to identify unresolved conditions and a responsible owner.
- Change review cadence
Link the recommendation to its evidence, remaining limits and next validation condition.
Define an experiment around an observed operating problem and a stable comparator. Record task quality, human review, rework, failures, and relevant system cost where permissible data exists. The proposed optimisation process should also retain a regression and rollback path so an apparent improvement can be challenged safely.
A practical first slice would focus on quality-adjusted outcomes and then follow its consequences into experiment isolation. Keep regression and rollback readiness visible as a separate review question rather than assuming it is satisfied by the first result. The customer leadership team should record observations, competing explanations, and changes to the original assumptions. A second reviewer can challenge whether the evidence supports the recommendation and whether an apparently successful example omitted the condition that matters most.
Indicative artefacts
Use change review cadence to make the next action explicit
Every artefact needs an operating use
Illustrative working format
- Optimisation backlog
- Boundary — operating baseline.
- Measurement and experiment plan
- Findings about experiment isolation.
- Change review cadence
- Action on regression and rollback readiness.
Indicative outputs include optimisation backlog, measurement and experiment plan, and change review cadence. Their form should follow the decision and the customer’s existing operating records. The first should establish the assessed boundary and relevant observations; the second should make material gaps or dependencies inspectable; the third should identify the action, owner, and evidence needed to revisit the decision.
Acceptance should be tested through use: can the responsible owner explain what was examined, identify an unresolved issue, and act on the next step without reconstructing the whole engagement? Findings should preserve counterevidence and limits. A polished document is insufficient if it does not help resolve the customer’s specific question.
Illustrative scenario
Drafts need staff correction: continue conditionally
Examine the constructed decision boundary
Illustrative scenario
- Drafts need staff correction
A regularly used support agent still requires human work before completion.
- Comparable-case experiment
Separate drafts from accepted work and target one recurring correction.
- Continue conditionally
Retain changes only if learning is useful and the team can maintain them.
Consider an illustrative situation in which a support agent is used regularly, but staff correct its outputs before completing the task. This is a constructed example, not a client engagement or an observed Softlander outcome.
For the corrected support outputs, an illustrative baseline would distinguish agent-generated drafts from accepted completed work. A proposed change might target one recurring correction type and then compare equivalent cases. The collaboration should continue only if its experiments produce decision-useful learning and fit the team’s capacity to implement and maintain changes.
The useful learning would be the difference between the expected behaviour and the behaviour or responsibility actually demonstrated. If the exercise leaves regression and rollback readiness unresolved, that should remain a named next step. A recommendation to narrow the scope, improve a prerequisite, or hold the decision can be appropriate when the example exposes a material dependency.
Scope and authority
A proposed mandate needs confirmed authority and capacity
Keep analytical work separate from decision authority
- Operating mandate and handover design
- Can include optimisation backlog.
- Executive sponsor
- Customer executives retain investment, release and risk decisions.
- Staffing, price and retained commitments
- Cannot be inferred from measurement and experiment plan.
These are proposed working arrangements, not claims of current staffing or an active retained relationship. Executives retain investment, release, and risk decisions. Capacity, role authority, conflicts, and commercial terms need explicit agreement before a commitment.
Ways to engage
Related engagement scopes
AI Fractional Agentic AI Office
Create a regular place for agentic AI decisions. Connect initiative priorities, architecture questions, risk ownership and evidence reviews.
AI Fractional Agentic Transformation Office
Coordinate the operating changes behind agent adoption. Align workflow redesign, ownership, dependencies and adoption measures.
AI Harness Engineering Pod
Engineer a bounded AI execution harness with tool adapters, context handling, orchestration, recovery and observable traces.
Continue exploring
Related insights and resources
Engineer the harness, not just the prompt
Define the contracts around an agent's decisions: what state it trusts, which actions can execute, how interrupted work resumes, and what evidence an operator needs to recover safely.
Governing agent decisions
Governance becomes useful when it changes a decision. Connect each agent workflow to an accountable owner, explicit operating conditions, proportionate evidence and a practical route for exceptions and withdrawal.
Do you need an agent centre of excellence?
A centre can concentrate scarce expertise and reusable controls, or become an approval queue detached from real work. Choose the mandate before the structure.