Evaluation & assurance5 MIN READ
Your agent passed the test. What does that prove?
A release decision needs a traceable connection between expected behaviour, test conditions and observed results. Here is a practical way to build that connection.
THE SOFTLANDER BLOG
Release decisions
Criteria, evidence and accountable judgement used to decide whether an agent should enter or remain in service.
In Release decisions
12 articlesArchitecture & engineering6 MIN READ
A CTO’s checklist before an agent reaches production
Production readiness for an AI agent spans workflow requirements, system identity, evaluation, recovery and operations—not just a working model endpoint.
Strategy & transformation6 MIN READ
How to write an RFP for agentic AI without buying a promise
Specify the workflow, evidence, responsibility and exit conditions suppliers must meet, then evaluate them through representative scenarios.
Impact measurement6 MIN READ
How to estimate the cost of an agent failure
Model concrete failure scenarios across detection, containment, correction, disruption and consequence, then use ranges to choose controls.
Evaluation & assurance6 MIN READ
Product acceptance criteria for non-deterministic systems
Accept an agent against outcome distributions, critical invariants and recovery evidence—not one scripted response or an average score.
Evaluation & assurance6 MIN READ
What product leaders should demand from an agent evaluation
Product leaders need evaluation evidence that connects workflow outcomes, user experience and policy behaviour to a release choice—not a leaderboard score.
Evaluation & assurance6 MIN READ
Turning red-team findings into release criteria
A dramatic exploit is a discovery, not a decision rule. Translate it into a threat condition, required control, repeatable regression and release consequence.
Strategy & transformation6 MIN READ
Build, buy or partner for an AI agent? Ask these questions first
Choose a sourcing path by separating the workflow advantage you must own from the capabilities you can responsibly obtain elsewhere.
Evaluation & assurance6 MIN READ
What confidence is enough for an agent release decision?
Confidence is not a universal percentage. It is a judgement about evidence quality, uncertainty, consequence and the reversibility of the proposed release.
Evaluation & assurance6 MIN READ
Beyond pass or fail: how to grade agent evaluation findings
Grade findings by consequence, reach, reproducibility and control strength, then keep that technical judgement separate from the release decision.
Evaluation & assurance6 MIN READ
What belongs in an AI agent release evidence pack?
A useful release pack connects the intended workflow, evaluated behaviour, unresolved risk and operating controls to one accountable decision.
Evaluation & assurance6 MIN READ
Design agent evaluations backwards from the decision
Start with the choice an owner must make, then derive claims, evidence, cases and measures. The result is smaller, sharper and easier to defend.
KEEP EXPLORING
Take the thinking
into your next discussion.
Read the method, assumptions and questions to ask.