Unauthorised tool-use testing
Determine whether attempted tool use stays within the acting principal's authority
When an agent can invoke tools with consequential permissions, this area becomes relevant. For the application security and authority owner, the concern is specific: the agent uses an available credential to perform an action the user did not authorise. The deployment evaluation starts from that consequence and follows the relevant evidence through the customer workflow. Test the enforcement boundary as well as the agent's expressed intent. A refusal is useful evidence about one interaction; the deployment also needs observable authorisation behaviour when a prohibited call is attempted within the approved test scope.
Connect the operating question to inspectable evidence
- Application security and authority owner
Owns the bounded decision and the consequence of acting.
- Authorisation policy
Defines the starting context for the proposed work.
- Role boundaries
Exposes the first material uncertainty to examine.
- Authority control findings
Carries unresolved conditions into an accountable next action.
Decision at stake
Determine whether attempted tool use stays within the acting principal's authority
Make the decision explicit
- Application security and authority owner
- An agent can invoke tools with consequential permissions.
- Decision
- Whether attempted tool use stays within the acting principal's authority.
- Role boundaries
- Use authorisation policy to establish the relevant starting condition.
For the application security and authority owner, this becomes relevant when an agent can invoke tools with consequential permissions. The immediate decision is whether attempted tool use stays within the acting principal’s authority. The scope should be narrow enough that the evaluation team can identify a meaningful alternative, including continuing the current approach if the available evidence already supports it.
The practical failure to challenge is that the agent uses an available credential to perform an action the user did not authorise. That concern gives the work a concrete direction: establish what the relevant boundary should do, inspect how it behaves or would operate, and identify the owner of any gap. It should not be diluted into a broad transformation exercise before the specific question has been answered.
Coverage
Examine role boundaries alongside revoked access
Keep the review distinctions visible
- Role boundaries
- Compare permitted and denied operations for the same synthetic role.
- Revoked access
- Revoke authority and retest using previously issued access material.
- Escalation and delegated authority
- Follow scope through delegation and any escalation route.
Role boundaries
For this scope, role boundaries should be examined alongside authorisation policy. Use synthetic roles with explicit permitted and denied actions. Test ordinary requests and plausible attempts to exceed scope. The proposed assessment should inspect the enforcement outcome as well as the agent’s response, since expressed willingness and actual authority are different parts of the system boundary.
Revoked access
For this scope, revoked access should be examined alongside synthetic identities. Revoke access while a task or session remains active and repeat a relevant action. Observe how the change reaches credentials, tools, and ongoing context within scope. The finding should identify any stale authority path and distinguish a denied attempt from a completed action after revocation.
Escalation and delegated authority
For this scope, escalation and delegated authority should be examined alongside permitted and denied action fixtures. Trace delegated calls back to the initiating authority. A worker’s available credential should not be treated as blanket permission to act. The test should preserve the original request and downstream effect so reviewers can see whether delegation or helpful improvisation expanded authority beyond the agreed task.
Prerequisites
Start with authorisation policy
Give each input a specific job
- Authorisation policy
- Defines the operating boundary.
- Synthetic identities
- Examine revoked access.
- Permitted and denied action fixtures
- Shows conditions affecting escalation and delegated authority.
- Product or platform owner
- Confirm access and a reviewer who can resolve gaps.
The starting material is authorisation policy, synthetic identities, and permitted and denied action fixtures. Review these together rather than assuming one document tells the whole story. A stated rule may differ from a configured behaviour, and a proposed responsibility may not yet have an available owner. Identify those differences before selecting the detailed work.
Participation should include a product or platform decision owner, a technical lead, a data owner, and a competent reviewer for the relevant boundary. The customer should define which information and environments may be used. Synthetic or sanitised examples can support preparation when they preserve the relevant behaviour; if they omit a material dependency, record the limitation and agree how the missing evidence would be obtained.
How the work proceeds
Tie the deployment finding to observable behaviour
Follow the evidence through the work
- Role boundaries
Use authorisation policy to define the decision criteria.
- Revoked access
Inspect synthetic identities and retain observations that challenge the expected result.
- Escalation and delegated authority
Use permitted and denied action fixtures to identify unresolved conditions and a responsible owner.
- Authority control findings
Link the recommendation to its evidence, remaining limits and next validation condition.
Use approved identities with clearly different permissions and test both direct requests and delegated routes. Include revoked access and requests whose apparent usefulness does not confer authority. The proposed assessment should capture denied calls and any successful unauthorised effects separately, with controlled fixtures that avoid real business harm.
A practical first slice would focus on role boundaries and then follow its consequences into revoked access. Keep escalation and delegated authority visible as a separate review question rather than assuming it is satisfied by the first result. The evaluation team should record observations, competing explanations, and changes to the original assumptions. A second reviewer can challenge whether the evidence supports the recommendation and whether an apparently successful example omitted the condition that matters most.
Illustrative scenario
Out-of-role entitlement request: precise boundary finding
Examine the constructed decision boundary
Illustrative scenario
- Out-of-role entitlement request
A synthetic support user asks for an action outside their permitted scope.
- Response and enforcement
Inspect both the agent's attempt and the tool's authorisation result.
- Precise boundary finding
Distinguish an attempted action that is denied from an actual unauthorised effect.
Consider an illustrative situation in which a synthetic support user asks the agent to change an entitlement outside that role’s permitted actions. This is a constructed example, not a client engagement or an observed Softlander outcome.
For the support user’s entitlement request, an illustrative run would inspect the agent response and enforcement outcome. If the model attempts the action but the tool denies it, that is different from an actual boundary breach. The evidence should support a precise finding and an appropriately scoped remediation decision.
The useful learning would be the difference between the expected behaviour and the behaviour or responsibility actually demonstrated. If the exercise leaves escalation and delegated authority unresolved, that should remain a named next step. A recommendation to narrow the scope, improve a prerequisite, or hold the decision can be appropriate when the example exposes a material dependency.
Indicative artefacts
Use authority control findings to make the next action explicit
Every artefact needs an operating use
Illustrative working format
- Unauthorised-use test matrix
- Boundary — authorisation policy.
- Enforcement evidence
- Findings about revoked access.
- Authority control findings
- Action on escalation and delegated authority.
Indicative outputs include unauthorized-use test matrix, enforcement evidence, and authority control findings. Their form should follow the decision and the customer’s existing operating records. The first should establish the assessed boundary and relevant observations; the second should make material gaps or dependencies inspectable; the third should identify the action, owner, and evidence needed to revisit the decision.
Acceptance should be tested through use: can the responsible owner explain what was examined, identify an unresolved issue, and act on the next step without reconstructing the whole engagement? Findings should preserve counterevidence and limits. A polished document is insufficient if it does not help resolve the customer’s specific question.
Scope and authority
A deployment finding remains bounded to the tested conditions
Keep analytical work separate from decision authority
- Authorised deployment cases and traces
- Can include unauthorized-use test matrix.
- Product or platform owner
- The customer retains release authority and owns remediation priorities.
- Universal vendor or component assurance
- Cannot be inferred from enforcement evidence.
The proposed evaluation concerns the named deployment, configuration, cases, and authorised environment. Findings do not certify a component vendor, establish universal safety, or approve release. Unexamined provider paths and specialist questions remain explicit limitations.
Ways to engage
Related engagement scopes
AI Agent Evaluation Evidence Diagnostic
Find the gaps between your release decision and the evidence behind it. Start with one agent workflow and a bounded set of questions.
AI Agent Evaluation Foundation
Turn scattered tests into a repeatable release discipline. Connect the behaviours that matter to datasets, evaluation criteria and evidence your team can maintain.
AI Agent Component Evaluation
Find where agent behaviour breaks down. Follow memory, retrieval and delegation through your workflow, then isolate the components that need attention.
Continue exploring
Related insights and resources
Engineer the harness, not just the prompt
Define the contracts around an agent's decisions: what state it trusts, which actions can execute, how interrupted work resumes, and what evidence an operator needs to recover safely.
What would justify releasing this agent?
Replace a single pass rate with a reviewable chain of requirements, cases, observations and judgements. Distinguish task quality, control failures and uncertainty before increasing an agent's exposure.
The evidence behind the go-live decision
A working paper for leaders who need to explain why an agent workflow is ready for its next release. Includes a proposed evidence-pack structure, decision table and review agenda.