Measuring AI impact
Measure what changed after the work, costs and corrections.
Agent activity is only part of the picture. Connect workflow outcomes to a credible baseline, human effort, operating costs and uncertainty before drawing a conclusion about value.
Measure what changed after the work, costs and corrections
- Baseline workflow
- Records performance before the change and how it was measured.
- Quality-adjusted outcome
- Checks completed work against the required standard.
- Full effort and cost
- Includes operation, review, correction and adoption work.
- Attribution limit
- Separates observed change from assumptions about its cause.
Discipline model
Count outcome quality and transferred effort together
The relationships that define the discipline
- Baseline workflow
- The comparison begins with a repeatable description of the original work.
- Quality-adjusted outcome
- Quality criteria distinguish useful completion from activity requiring correction.
- Full effort and cost
- New review and operating work must be considered alongside apparent savings.
- Attribution limit
- Confounding changes limit how much of the difference can be attributed to AI.
More messages, tool calls or completions do not establish a better business result. Define the outcome in the workflow’s own terms and record the costs introduced elsewhere. Faster first drafts may create more review; apparently avoided work may reappear as correction or supervision. A useful measurement design makes these transfers visible.
Working approach
Choose a comparison the evidence can support
Proposed investigation sequence
- Define baseline workflow
State the decision and conditions the work must address.
- Map quality-adjusted outcome
Connect the relevant actors, configuration and evidence.
- Examine full effort and cost
Use agreed evidence and controlled cases where appropriate.
- Review attribution limit
Record limits, ownership and the condition for a next decision.
Agree the outcome and comparison method with the workflow owner and finance. Capture a baseline where possible, define data sources and quality criteria, and account for operating, review, failure and adoption costs. Report uncertainty and confounding changes. When the available evidence cannot support causal attribution, describe the observed association and what further measurement is needed.
Working artefact
Every value entry needs a method and an owner
Illustrative working record
Illustrative working format
- Baseline workflow
- Define the subject and the requirement the record addresses.
- Evidence reference
- Link source, version and observed condition without copying sensitive material.
- Full effort and cost
- Explain the finding or choice and its consequence.
- Attribution limit
- Assign the next action and state what remains unresolved.
A value-model entry needs a metric definition, unit, data source, observation period, comparison and limitations. Assumptions should be distinct from observations, with owners responsible for updating them. Reporting should support a decision to continue, change or stop the initiative rather than optimise a headline benefit number.
Illustrative situation
More completions can bring more correction work
A constructed situation to reason through
Illustrative scenario
- Activity increases
An agent completes more draft cases during a trial.
- Review expands
People spend additional effort correcting those cases.
- Value question
Compare acceptable final outcomes and total effort with the baseline.
Use this situation to identify the evidence that would change the decision. The proposed examination must stay within the agreed scope, and any conclusion must describe the conditions actually inspected.
Scope and next decision
Unsupported benefit estimates remain assumptions
Prepare a bounded next discussion
- Baseline workflow
- Bring one concrete workflow question and its accountable owner.
- Quality-adjusted outcome
- Identify the evidence and access that can be made available.
- Full effort and cost
- Agree the specialist involvement and responsibilities required.
- Written scope
- Confirm activities, evidence, readiness and acceptance before commitment.
No savings, revenue uplift or return is implied without measurement evidence. Financial models provide management decision support; they do not replace authoritative accounting records or specialist financial advice. Missing data should remain unknown rather than becoming zero cost.
Explore the discipline
Specialist areas within this capability
Choose a question to explore its methods, boundaries and indicative outputs.
- AI value measurement design
- AI program baseline assessment
- Productivity-impact measurement
- Workflow outcome measurement
- AI quality-adjusted ROI
- Human-effort and review-cost measurement
- Failure-cost modelling
- AI total-cost-of-ownership analysis
- Agent unit-economics assessment
- Adoption and utilization measurement
- Customer-experience impact
- Employee-experience impact
- Risk-adjusted value measurement
- Time-to-decision measurement
- Revenue-attribution design
- Cost-avoidance validation
- AI portfolio value realization
- AI impact dashboard design
- Benefits-realization assurance
- Independent AI investment review
Related services
AI Impact Measurement Framework
Define what better should mean before scaling an agent. Connect quality, cost, adoption and business outcomes to a baseline your team can revisit.
Relevant insights
Measure the work that improves
Connect agent activity to accepted business outcomes, establish a credible comparison, and distinguish useful capacity from realized financial value before deciding whether to scale.
Before the pilot: five questions for agent readiness
A compelling demo leaves important questions open. Use a five-dimension rubric to examine the data, authority, evidence, ownership and people behind your next agent initiative.
Where smaller models can strengthen agent architecture
Smaller models are not merely budget substitutes. Used behind explicit routing and escalation rules, they can improve latency, locality and architectural control.