Model-routing architecture
Decide when routing among models is justified by observed task requirements
When one model is being used for tasks with different needs, this area becomes relevant. For the AI platform and cost owner, the concern is specific: a lower-cost route is chosen without checking whether human rework increased. The engineering design starts from that consequence and follows the relevant evidence through the customer workflow. The route should be an explicit decision with inspectable inputs. Assess task quality, latency, cost, and failure handling together, using the customer's workload rather than assuming a model hierarchy applies universally.
The engineering decisions relationship
- Task categories
Establish the basis for route eligibility.
- Equivalent comparison
Hold quality and control constraints constant.
- Routing policy design
Retain the scoped finding and its unresolved dependencies.
Decision
Frame the decision around route eligibility
Questions and the evidence they need
- Route eligibility
- Inspect task categories for the relevant constraint or observation.
- Equivalent comparison
- Inspect candidate model configurations for the relevant constraint or observation.
- Fallback behaviour
- Inspect quality and operating constraints for the relevant constraint or observation.
For the AI platform and cost owner, this becomes relevant when one model is being used for tasks with different needs. The immediate decision is when routing among models is justified by observed task requirements. The scope should be narrow enough that the engineering team can identify a meaningful alternative, including continuing the current approach if the available evidence already supports it.
The practical failure to challenge is that a lower-cost route is chosen without checking whether human rework increased. That concern gives the work a concrete direction: establish what the relevant boundary should do, inspect how it behaves or would operate, and identify the owner of any gap. It should not be diluted into a broad transformation exercise before the specific question has been answered.
Mechanism
Model routing should preserve the task's acceptance criteria
Relationships within engineering decisions
- Route eligibility
- Match model choice to observed task requirements.
- Equivalent comparison
- Hold quality and control constraints constant.
- Fallback behaviour
- Specify what changes when the preferred route is unavailable.
Routing eligibility
For this scope, routing eligibility should be examined alongside task categories. Define which tasks and data contexts are eligible for each route. A route may be unsuitable despite acceptable quality on another task. The proposed policy should preserve constraints such as required capabilities, permitted data handling, and action consequences before considering efficiency.
Fallback behaviour
For this scope, fallback behaviour should be examined alongside candidate model configurations. Specify fallback conditions and what remains authorised when the preferred route fails. A fallback should not silently relax a task requirement or control boundary. The architecture should record the reason for selection so operators can explain why a different model handled a particular case.
Comparison on equivalent tasks
For this scope, comparison on equivalent tasks should be examined alongside quality and operating constraints. Compare equivalent tasks using agreed quality, latency, cost, and recovery criteria. Record missing operating inputs and available human correction effort. A lower local model cost should remain a hypothesis about overall value until the completed workflow and its rework are examined.
Preparation
Connect the working inputs to the boundary being examined
Evidence packet for route eligibility
Illustrative working format
- Task categories
- Identify provenance and the specific decision this material can inform.
- Candidate model configurations
- Match this description to the deployed or proposed boundary being examined.
- Quality and operating constraints
- Name the custodian and the conditions requiring a refreshed version.
The starting material is task categories, candidate model configurations, and quality and operating constraints. Review these together rather than assuming one document tells the whole story. A stated rule may differ from a configured behaviour, and a proposed responsibility may not yet have an available owner. Identify those differences before selecting the detailed work.
Participation should include a workflow owner, an implementation lead, an eventual maintainer, and qualified control reviewers where needed. The customer should define which information and environments may be used. Synthetic or sanitised examples can support preparation when they preserve the relevant behaviour; if they omit a material dependency, record the limitation and agree how the missing evidence would be obtained.
How the work proceeds
Carry route eligibility through to an accountable finding
From the scoped material to a decision record
- Task categories
Make the component and operating boundary explicit.
- Equivalent comparison
Apply the scoped check to fallback behaviour.
- Routing policy design
Record the evidence, open dependencies, and the owner’s next decision.
Define routing eligibility before optimising it. Some routes may be disallowed by data handling, required capabilities, or action consequences even if they perform well on another metric. The proposed routing policy should preserve the reason for selection and identify what happens when the chosen route is unavailable or fails its acceptance checks.
A practical first slice would focus on routing eligibility and then follow its consequences into fallback behaviour. Keep comparison on equivalent tasks visible as a separate review question rather than assuming it is satisfied by the first result. The engineering team should record observations, competing explanations, and changes to the original assumptions. A second reviewer can challenge whether the evidence supports the recommendation and whether an apparently successful example omitted the condition that matters most.
Indicative artefact
Make the routing policy design usable by its owner
Routing policy design
Illustrative working format
- Routing policy design
- Identify the governing revision, exception path, and accountable interpretation owner.
- Comparative evaluation plan
- Link supporting and contrary observations to inspectable evidence.
- Fallback and monitoring specification
- Name the unresolved condition, next action, and accountable owner.
Indicative outputs include routing policy design, comparative evaluation plan, and fallback and monitoring specification. Their form should follow the decision and the customer’s existing operating records. The first should establish the assessed boundary and relevant observations; the second should make material gaps or dependencies inspectable; the third should identify the action, owner, and evidence needed to revisit the decision.
Acceptance should be tested through use: can the responsible owner explain what was examined, identify an unresolved issue, and act on the next step without reconstructing the whole engagement? Findings should preserve counterevidence and limits. A polished document is insufficient if it does not help resolve the customer’s specific question.
Illustrative scenario
Examine fallback behaviour through a constructed exception
A constructed case and its decision paths
Illustrative scenario
- Route eligibility
Classification, evidence synthesis, and action preparation use different model routes.
- Equivalent comparison
Compare routes on equivalent tasks and an unavailable preferred model.
- Fallback behaviour
Preserve required quality and controls in both normal and fallback routing.
Consider an illustrative situation in which a service workflow uses different model routes for classification, evidence synthesis, and action preparation. This is a constructed example, not a client engagement or an observed Softlander outcome.
For the classification-and-synthesis example, the illustrative comparison would use equivalent cases and record both intermediate decisions and completed outcomes. A route that performs well on classification might still be unsuitable for preparing a consequential action. The architecture decision should remain specific to the task category and operating constraints tested.
The useful learning would be the difference between the expected behaviour and the behaviour or responsibility actually demonstrated. If the exercise leaves comparison on equivalent tasks unresolved, that should remain a named next step. A recommendation to narrow the scope, improve a prerequisite, or hold the decision can be appropriate when the example exposes a material dependency.
Scope and next action
Keep the conclusion within the defined workflow and configuration
Confirm fit before the next commitment
- Route eligibility
- This review is limited to the defined workflow and configuration.
- Task categories
- The relevant owner must confirm access and the intended operating conditions.
- Routing policy design
- Production acceptance needs separate scoped evidence.
The method described here is proposed engineering work. Scope, staffing, implementation access, acceptance, and commercial terms require a readiness review. A design or lab result does not establish production suitability, guaranteed reliability, or security certification.
Ways to engage
Related engagement scopes
AI Fractional Agentic AI Office
Create a regular place for agentic AI decisions. Connect initiative priorities, architecture questions, risk ownership and evidence reviews.
AI Fractional Agentic Transformation Office
Coordinate the operating changes behind agent adoption. Align workflow redesign, ownership, dependencies and adoption measures.
AI Harness Engineering Pod
Engineer a bounded AI execution harness with tool adapters, context handling, orchestration, recovery and observable traces.
Continue exploring
Related insights and resources
Engineer the harness, not just the prompt
Define the contracts around an agent's decisions: what state it trusts, which actions can execute, how interrupted work resumes, and what evidence an operator needs to recover safely.
Governing agent decisions
Governance becomes useful when it changes a decision. Connect each agent workflow to an accountable owner, explicit operating conditions, proportionate evidence and a practical route for exceptions and withdrawal.
Do you need an agent centre of excellence?
A centre can concentrate scarce expertise and reusable controls, or become an approval queue detached from real work. Choose the mandate before the structure.