Evaluation & assurance5 MIN READ
Your agent passed the test. What does that prove?
A release decision needs a traceable connection between expected behaviour, test conditions and observed results. Here is a practical way to build that connection.
THE SOFTLANDER BLOG
Evaluation evidence
Tests, observations and decision records that show how an agent behaves under relevant conditions.
In Evaluation evidence
31 articlesGovernance & privacy7 MIN READ
What should trigger an agent governance review?
Review should follow material changes in purpose, authority, data, users, dependencies, evidence or incidents—not an arbitrary calendar alone.
Governance & privacy7 MIN READ
You cannot govern AI systems you have not inventoried
An AI inventory becomes useful when it identifies deployed systems, their context, owners, dependencies and review triggers—not merely a list of model vendors.
Evaluation & assurance6 MIN READ
Golden traces are useful—but they are not gold standards
A reference trace can explain one acceptable journey. Treating it as the only correct path can punish valid behaviour and conceal unsafe alternatives.
Impact measurement6 MIN READ
How to estimate the cost of an agent failure
Model concrete failure scenarios across detection, containment, correction, disruption and consequence, then use ranges to choose controls.
Evaluation & assurance6 MIN READ
How to design a joint AI evaluation lab
A joint lab should be a governed way to create decision evidence across organisations, with shared methods but explicit ownership of systems, data and conclusions.
Strategy & transformation6 MIN READ
What public-sector buyers should request from an agent supplier
Buy evidence, controllability and lifecycle obligations—not a confident demo or an unrestricted promise of autonomy.
Evaluation & assurance6 MIN READ
Ten artefacts that make an AI-agent decision reviewable
Reviewable agent decisions depend on a small chain of artefacts connecting workflow purpose, authority, configuration, evidence, findings, controls and ownership.
Governance & privacy7 MIN READ
Can your agent truly forget?
Deleting a chat is not the same as removing personal data from memory, retrieval, logs, caches, evaluations, backups and learned models.
Evaluation & assurance6 MIN READ
Measure the action, not just the answer
A convincing message can conceal a failed or unauthorised operation. Agent evaluation must inspect tool calls, external state and control decisions.
Impact measurement6 MIN READ
How to write an AI success story that survives scrutiny
A credible AI success story separates what was delivered from what changed, defines the baseline, shows the contribution chain and preserves uncertainty and permission.
Governance & privacy7 MIN READ
Human oversight is a designed control, not a person watching a screen
Effective oversight gives a prepared person the signal, authority, time and interface needed to prevent or limit a defined AI failure.
KEEP EXPLORING
Take the thinking
into your next discussion.
Read the method, assumptions and questions to ask.