Agent unit-economics assessment
Measure agent economics per accepted business outcome
Agent unit-economics assessment becomes relevant when cost-per-call dashboards do not explain whether useful work is economically viable. A focused review would examine a completed, accepted workflow unit including unsuccessful attempts and required human work. This is most relevant to product finance partners and platform product owners, especially where teams must distinguish cost per request and cost per accepted task.
A measure needs a unit, comparator and decision
Illustrative working format
- Measurement unit
- A completed, accepted workflow unit including unsuccessful attempts and required human work.
- Execution and provider usage records
- Establish the starting evidence and its measurement boundary.
- Cost per accepted task
- Keep this distinct when interpreting the proposed measure.
- A design and operating-economics brief
- Connect uncertainty to the management choice it can change.
Decision at stake
Measure agent economics per accepted business outcome
Make the decision explicit
- Product finance partners and platform product owners
- Cost-per-call dashboards do not explain whether useful work is economically viable.
- Decision
- What it costs to complete an acceptable unit of work and which design choices affect that cost.
- Cost per request
- Use execution and provider usage records to establish the relevant starting condition.
For product finance partners and platform product owners, the immediate question is what it costs to complete an acceptable unit of work and which design choices affect that cost. This is most useful when there is a real choice between proceeding, changing the design, collecting more evidence or narrowing the intended use. The starting conversation should identify that choice and its consequence before deciding how much work the topic warrants.
An inexpensive model call could be part of an expensive task when repeated attempts and manual correction are included. A useful review follows a completed, accepted workflow unit including unsuccessful attempts and required human work, rather than stopping at a convenient component metric or document boundary. The evidence should separate observation, interpretation and recommendation so the accountable owner can see what it supports.
Prerequisites
Start with execution and provider usage records
Give each input a specific job
- Execution and provider usage records
- Defines the operating boundary.
- Completion, retry and quality dispositions
- Examine marginal cost.
- Review effort and shared-cost rules
- Shows conditions affecting successful run cost.
- Business and finance owner
- Confirm access and a reviewer who can resolve gaps.
For Agent unit-economics assessment, the sponsor should identify the decision, authoritative data sources and the owner of each measure. A baseline or credible comparator is especially important. Where the relevant information is missing, the work should establish a prospective measurement plan or narrow the claim rather than manufacture historical precision.
Useful starting material would include execution and provider usage records; completion, retry and quality dispositions; review effort and shared-cost rules. These inputs have different jobs: the first establishes context, the second supplies behaviour or comparison material, and the third exposes operating conditions that may qualify a conclusion. Existing material would be reviewed before requesting new collection, and a missing input would be recorded as an evidence gap rather than treated as a favourable result.
Coverage
Cost per request and cost per accepted task need different evidence
Keep the review distinctions visible
- Cost per request
- Report separately from cost per accepted task.
- Marginal cost
- Report separately from fully allocated cost.
- Successful run cost
- Report separately from total attempt cost.
Cost per request versus cost per accepted task
The denominator would be accepted business work, not model calls. Resources consumed by retries, abandoned tasks and corrections would remain in the workload cost so success-path efficiency does not understate operation.
Marginal cost versus fully allocated cost
Cost attribution would trace model, tool and human activity to a case where practical. Shared infrastructure and support would be handled through explicit allocation rather than silently omitted or assigned twice.
Successful run cost versus total attempt cost
Design comparisons would test whether lower per-call costs change completion or review burden. A cheaper component can be an attractive option, but its effect on accepted-task economics needs a matched workflow comparison.
Illustrative scenario
Accepted research report: examine what changes the decision
Examine the constructed decision boundary
Illustrative scenario
- Accepted research report
Multiple retrieval attempts and a human reviewer contribute to one usable report.
- Abandoned attempts
Retain unsuccessful work in the relevant effort and cost allocation.
- Unit comparison
Compare cost per accepted task instead of cost per model request.
In a fictional account-research workflow, an agent makes several retrieval and model calls before a reviewer accepts a report. Some tasks are abandoned after repeated retries. The proposed assessment would allocate relevant effort and cost across accepted work and preserve the abandoned attempts. It would help compare design changes without equating lower token cost with better unit economics.
The accepted-report cost could vary substantially between routine and ambiguous accounts. A single average would hide that difference, so the proposed assessment would preserve the strata. The product owner could then test routing or review changes and compare their effect on accepted outcomes rather than on isolated call efficiency. These are hypothetical decision paths, not observed client outcomes.
How the work proceeds
Define the comparison before interpreting the value
Follow the evidence through the work
- Execution and provider usage records
Define the unit, period and comparator before interpreting a change.
- Completion, retry and quality dispositions
Inspect this evidence for missing cases and incompatible definitions.
- Marginal cost
Keep this distinction visible when assessing the apparent value.
- A design and operating-economics brief
Show assumptions and the observation that could reverse the decision.
Define the measurement
Define the accepted outcome unit and attribution boundary. The opening step turns the operating concern into a question that can be examined. It would identify a completed, accepted workflow unit including unsuccessful attempts and required human work, the outcome the owner cares about and the conditions that would make an observation decision-relevant. Scope changes would remain explicit if the evidence reveals a different problem.
Check the evidence
Join execution, quality, retry and human-effort records. The comparison material would include completion, retry and quality dispositions, with review effort and shared-cost rules available to explain the operating conditions. Differences between cost per request and cost per accepted task would remain visible beside the observation rather than disappear into a pooled result.
Examine alternatives
Separate marginal and allocated cost views with explicit assumptions. The review question would explicitly distinguish marginal cost from fully allocated cost. A material discrepancy would be traced back to execution and provider usage records and the relevant execution or decision record, so a proposed change addresses the observed cause rather than the most convenient explanation.
Support the decision
Compare task strata and proposed changes through sensitivity analysis. The resulting design and operating-economics brief would explain the distinction between successful run cost and total attempt cost and identify what further observation would change the recommendation. The business and finance sponsor would retain the decision, including whether a narrower scope or a different approach is preferable.
Indicative artefacts
Use a design and operating-economics brief to make the next action explicit
Every artefact needs an operating use
Illustrative working format
- An accepted-outcome unit-cost model
- Boundary — execution and provider usage records.
- A cost and failure contribution breakdown
- Findings about marginal cost.
- A design and operating-economics brief
- Action on successful run cost.
The final artefact set depends on the agreed decision. Useful outputs can include:
- An accepted-outcome unit-cost model. The working foundation for the question, including relevant definitions, boundaries and assumptions so another reviewer can understand what was examined.
- A cost and failure contribution breakdown. The substantive observations and interpretation, preserving adverse results, disagreement and limitations that affect how the evidence may be used.
- A design and operating-economics brief. The connection to action: what the owner could do next, what evidence is still missing and what change would justify revisiting the conclusion.
Scope and authority
Measured evidence and assumptions have different roles in a funding decision
Keep analytical work separate from decision authority
- Defined measures and management analysis
- Can include an accepted-outcome unit-cost model.
- Business and finance owner
- Finance and business owners validate assumptions and retain funding decisions.
- Audited returns or investment approval
- Cannot be inferred from a cost and failure contribution breakdown.
The proposed work supports management analysis and does not approve an investment or establish audited financial results. Prices, returns and realized benefits depend on actual evidence. Accounting, regulated advice and legal interpretations remain with qualified owners; authoritative financial and CRM records remain in their approved systems. To discuss agent unit-economics assessment, the first practical step is to identify the relevant owner and bring a sanitised description of the decision, the current alternative and the material uncertainty. That gives scoping a concrete starting point without implying a preapproved package or favourable conclusion.
Ways to engage
Related engagement scopes
AI Impact Measurement Framework
Define what better should mean before scaling an agent. Connect quality, cost, adoption and business outcomes to a baseline your team can revisit.
Continue exploring
Related insights and resources
Measure the work that improves
Connect agent activity to accepted business outcomes, establish a credible comparison, and distinguish useful capacity from realized financial value before deciding whether to scale.
Before the pilot: five questions for agent readiness
A compelling demo leaves important questions open. Use a five-dimension rubric to examine the data, authority, evidence, ownership and people behind your next agent initiative.
Where smaller models can strengthen agent architecture
Smaller models are not merely budget substitutes. Used behind explicit routing and escalation rules, they can improve latency, locality and architectural control.