How to test an AI agent before it touches production
Run it against real inputs with its writes going nowhere, compare what it proposed against what actually happened, then give it a small share of live traffic with spend caps and a fast stop. Stages without written promotion criteria are decoration.
Most agent testing plans are a demo and a hope. The agent is exercised by the people who built it, on inputs they thought of, and when it behaves it goes live with a note to watch it closely for the first week. Watching closely is not a test, and the first week is when the inputs are least representative.
The approach that works borrows from how anyone ships risky infrastructure: run the new thing against real traffic while it can do no harm, compare what it wanted to do with what actually happened, then let it act on a small slice with a fast way to stop it. What makes this work is not the stages. It is having written criteria for moving between them, because stages without criteria are decoration and everyone advances on schedule pressure instead.
Stage one: shadow mode
Run the agent against real production inputs with every write redirected to a store nobody reads. It proposes; it does not act. Real tickets, real emails, real orders, real weirdness.
The value is in the comparison. For each shadow decision, record what the agent proposed and what the human or existing system actually did, then measure agreement. Disagreements are the entire point, and they split three ways: the agent was wrong, the agent was right and the human was wrong, or both were defensible. That third bucket is usually larger than expected and is where you discover your task has no single correct answer, which changes what you can promise.
Shadow mode also gives you a real cost figure before you commit. As we cover in the build versus buy arithmetic, the number of actions a conversation actually takes is the thing that determines your bill, and shadow traffic measures it instead of estimating it.
Stage two: capped canary
Then let it act, on a small share of traffic, with limits that are enforced rather than intended.
- A share, not a pilot group. Route a percentage of ordinary traffic. Hand-picked pilot users are the friendliest inputs you have.
- A spend cap per task and per day, enforced at the gateway, that stops rather than alerts.
- A stop that one person can operate without a deploy, and that has been tested at least once on purpose. An untested kill switch is a belief.
- Constrained permissions for the canary, narrower than the eventual design. Widen them as evidence accumulates.
Keep it there longer than feels necessary. The failures worth catching are not in the first hundred interactions, they are in the tail: the malformed input, the customer who writes in two languages, the record with a null nobody expected.
The criteria between stages
This is the part that gets skipped, so write it down before you start, when nobody is under pressure to ship.
| Gate | What has to be true | Written down as |
|---|---|---|
| Shadow to canary | Agreement rate above a number you chose in advance, and every disagreement class understood rather than merely counted | A number and a review, dated |
| Canary to wider traffic | A period at volume with no incident in the categories you defined as serious, and cost per task within the estimate | A duration, an incident definition, a budget |
| Wider to full | Kill switch exercised, on call knows what to do, every prior incident has a regression case | A checklist someone signs |
Decide what counts as a serious incident before launch, in writing. Afterwards, every incident is discussed on its merits, and the conversation is about whether this particular one really counts. Deciding in advance is the only version of this that survives contact with a deadline.
Every incident becomes a test
The single highest return habit in this area: when something goes wrong, the fix is not complete until the exact input that caused it is a permanent case in your evaluation set, with the expected behaviour recorded.
Do this consistently and your evaluation set becomes the one thing that is genuinely specific to your business, made of real failures rather than imagined ones. It also gives you something to run when you change model, which you will, and which is otherwise a leap of faith. Teams that skip this rediscover the same three bugs annually.
Who decides, and when they decide it
The stages above fail for organisational reasons more often than technical ones, and the pattern is consistent: the person who can stop a launch is not the person watching the numbers.
Name one owner for each gate before the first shadow run, and give them the authority to hold it. That is not ceremony. Without it, promotion happens because a date arrived, and the criteria become something to explain away rather than something to meet. The most useful version we have seen is a named person, a written number, and a short standing meeting where the only question is whether the number was met.
What to measure that is not accuracy
- Agreement with the existing process, and the shape of the disagreements.
- Escalation rate. How often it hands off. Too low is more worrying than too high, because it means it is not recognising its own limits.
- Actions per task, which drives cost and tends to drift upward as prompts grow.
- Time to complete, including the tool calls, not just model latency.
- Rate of repeated failures on the same input class, which tells you whether your fixes generalise or are patches.
If you are about to put an agent in front of real systems and want the rollout plan and its promotion criteria written before rather than after, our free AI audit covers what the agent can currently reach, and how we evaluate AI systems covers the harness we build for it.
Related reading
Get this applied to your business.
The free AI audit measures your live setup and shows where AI would actually pay off.