Enterprise pilot factory
Decide how to run bounded pilots that produce comparable learning and stop decisions
When many business teams want to pilot agents, this area becomes relevant. For the innovation or transformation sponsor, the concern is specific: a pilot factory becomes a demo factory with no evidence of operational fit. The operating model design starts from that consequence and follows the relevant evidence through the customer workflow. The factory should standardise learning conditions rather than force every use case into the same solution. A completed pilot needs a decision about what was learned, whether adoption is justified, and what remains unproven.
Connect the operating question to inspectable evidence
- Innovation or transformation sponsor
- Owns the bounded decision and the consequence of acting.
- Candidate use cases
- Defines the starting context for the proposed work.
- Pilot scope discipline
- Exposes the first material uncertainty to examine.
- Transition decision template
- Carries unresolved conditions into an accountable next action.
Decision at stake
Decide how to run bounded pilots that produce comparable learning and stop decisions
Make the decision explicit
- Innovation or transformation sponsor
- Many business teams want to pilot agents.
- Decision
- How to run bounded pilots that produce comparable learning and stop decisions.
- Pilot scope discipline
- Use candidate use cases to establish the relevant starting condition.
For the innovation or transformation sponsor, this becomes relevant when many business teams want to pilot agents. The immediate decision is how to run bounded pilots that produce comparable learning and stop decisions. The scope should be narrow enough that the customer leadership team can identify a meaningful alternative, including continuing the current approach if the available evidence already supports it.
The practical failure to challenge is that a pilot factory becomes a demo factory with no evidence of operational fit. That concern gives the work a concrete direction: establish what the relevant boundary should do, inspect how it behaves or would operate, and identify the owner of any gap. It should not be diluted into a broad transformation exercise before the specific question has been answered.
Coverage
Examine pilot scope discipline alongside outcome measurement
Keep the review distinctions visible
- Pilot scope discipline
- Admit a pilot only with a bounded task and observable acceptance criteria.
- Outcome measurement
- Specify the baseline and outcome before interpreting pilot activity.
- Stop and transition decisions
- Require a recorded expand, redesign or stop decision at the pilot gate.
Pilot scope discipline
For this scope, pilot scope discipline should be examined alongside candidate use cases. Bound each pilot by one workflow, learning question, authority limit, and evidence plan. A pilot factory should standardise these decisions without forcing every use into one solution. Requests lacking an owner or usable comparator should remain in discovery rather than consume implementation capacity.
Outcome measurement
For this scope, outcome measurement should be examined alongside pilot eligibility criteria. Define outcomes and failure thresholds before the pilot runs. Separate participation, prototype completion, task quality, and operating value. The proposed method should preserve inconclusive results and access failures instead of relabeling them as evidence of demand or technical success.
Stop and transition decisions
For this scope, stop and transition decisions should be examined alongside available technical and review capacity. Specify advance, revise, hold, and stop decisions with owners and evidence needs. A completed pilot is useful when it changes the next investment decision. Transition to production requires additional readiness and operating proof; it should not follow automatically from finishing a demonstration.
Prerequisites
Start with candidate use cases
Give each input a specific job
- Candidate use cases
- Defines the operating boundary.
- Pilot eligibility criteria
- Examine outcome measurement.
- Available technical and review capacity
- Shows conditions affecting stop and transition decisions.
- Executive sponsor
- Confirm access and a reviewer who can resolve gaps.
The starting material is candidate use cases, pilot eligibility criteria, and available technical and review capacity. Review these together rather than assuming one document tells the whole story. A stated rule may differ from a configured behaviour, and a proposed responsibility may not yet have an available owner. Identify those differences before selecting the detailed work.
Participation should include an executive sponsor, accountable delivery owners, and the people who can accept or implement the resulting decisions. The customer should define which information and environments may be used. Synthetic or sanitised examples can support preparation when they preserve the relevant behaviour; if they omit a material dependency, record the limitation and agree how the missing evidence would be obtained.
How the work proceeds
Make the mandate usable in operating decisions
Follow the evidence through the work
- Pilot scope discipline
Use candidate use cases to define the decision criteria.
- Outcome measurement
Inspect pilot eligibility criteria and retain observations that challenge the expected result.
- Stop and transition decisions
Use available technical and review capacity to identify unresolved conditions and a responsible owner.
- Transition decision template
Link the recommendation to its evidence, remaining limits and next validation condition.
A pilot should have a question, comparator, operating boundary, and decision threshold before implementation begins. The proposed factory should also reserve capacity for evaluation and closeout. An initiative without a business owner, permissible evidence, or credible next decision should remain in discovery rather than consume delivery capacity.
A practical first slice would focus on pilot scope discipline and then follow its consequences into outcome measurement. Keep stop and transition decisions visible as a separate review question rather than assuming it is satisfied by the first result. The customer leadership team should record observations, competing explanations, and changes to the original assumptions. A second reviewer can challenge whether the evidence supports the recommendation and whether an apparently successful example omitted the condition that matters most.
Indicative artefacts
Use transition decision template to make the next action explicit
Every artefact needs an operating use
Illustrative working format
- Pilot operating design
- Boundary — candidate use cases.
- Eligibility and evidence rubric
- Findings about outcome measurement.
- Transition decision template
- Action on stop and transition decisions.
Indicative outputs include pilot operating design, eligibility and evidence rubric, and transition decision template. Their form should follow the decision and the customer’s existing operating records. The first should establish the assessed boundary and relevant observations; the second should make material gaps or dependencies inspectable; the third should identify the action, owner, and evidence needed to revisit the decision.
Acceptance should be tested through use: can the responsible owner explain what was examined, identify an unresolved issue, and act on the next step without reconstructing the whole engagement? Findings should preserve counterevidence and limits. A polished document is insufficient if it does not help resolve the customer’s specific question.
Illustrative scenario
Uneven departmental requests: different next steps
Examine the constructed decision boundary
Illustrative scenario
- Uneven departmental requests
One support workflow has an owner and baseline; another is only an idea.
- Pilot eligibility
Compare each request against bounded scope and observable learning conditions.
- Different next steps
Advance the defined workflow if eligible; route the vague idea to problem discovery.
Consider an illustrative situation in which several departments request agents, but only some have a workflow owner and usable baseline. This is a constructed example, not a client engagement or an observed Softlander outcome.
For the departmental requests, an illustrative selection exercise would compare a well-defined support workflow with a vague automation idea. The first might qualify for a bounded pilot while the second needs problem discovery. The factory’s repeatability comes from consistent learning gates, not from approving every request or deploying the same agent pattern everywhere.
The useful learning would be the difference between the expected behaviour and the behaviour or responsibility actually demonstrated. If the exercise leaves stop and transition decisions unresolved, that should remain a named next step. A recommendation to narrow the scope, improve a prerequisite, or hold the decision can be appropriate when the example exposes a material dependency.
Scope and authority
A proposed mandate needs confirmed authority and capacity
Keep analytical work separate from decision authority
- Operating mandate and handover design
- Can include pilot operating design.
- Executive sponsor
- Customer executives retain investment, release and risk decisions.
- Staffing, price and retained commitments
- Cannot be inferred from eligibility and evidence rubric.
These are proposed working arrangements, not claims of current staffing or an active retained relationship. Executives retain investment, release, and risk decisions. Capacity, role authority, conflicts, and commercial terms need explicit agreement before a commitment.
Ways to engage
Related engagement scopes
AI Fractional Agentic AI Office
Create a regular place for agentic AI decisions. Connect initiative priorities, architecture questions, risk ownership and evidence reviews.
AI Fractional Agentic Transformation Office
Coordinate the operating changes behind agent adoption. Align workflow redesign, ownership, dependencies and adoption measures.
AI Harness Engineering Pod
Engineer a bounded AI execution harness with tool adapters, context handling, orchestration, recovery and observable traces.
Continue exploring
Related insights and resources
Engineer the harness, not just the prompt
Define the contracts around an agent's decisions: what state it trusts, which actions can execute, how interrupted work resumes, and what evidence an operator needs to recover safely.
Governing agent decisions
Governance becomes useful when it changes a decision. Connect each agent workflow to an accountable owner, explicit operating conditions, proportionate evidence and a practical route for exceptions and withdrawal.
Do you need an agent centre of excellence?
A centre can concentrate scarce expertise and reusable controls, or become an approval queue detached from real work. Choose the mandate before the structure.