Where human approval actually belongs in an AI agent
In front of actions that cannot be undone, or that reach further than a blast radius you have deliberately chosen. Gating everything is close to gating nothing, because a control that fires constantly stops being read. The design work is deciding which actions qualify, then shrinking permissions so the list stays short.
"Human in the loop" appears in almost every agent proposal we see, including ones that are otherwise thoughtful. It is a phrase that ends conversations rather than answering them. A loop containing a human is not a control until somebody says which actions stop there, who that human is, and what they are expected to notice.
This piece follows on from constraining what an agent can do. Once you have decided which of an agent's actions are consequential, the next question is which of those a person should personally authorise. The answer is fewer than most teams assume, and choosing badly makes the system less safe rather than more.
Why gating everything fails
The intuition is that more approvals mean more safety, so when in doubt you gate. In practice, a control that fires on every action stops being read. The person clicking approve forty times an hour is not performing forty security reviews, they are clearing a queue, and by the fortieth they are not looking at the contents at all. You have not added oversight. You have added a delay, and convinced yourself it was oversight.
This is worth being honest about, because the alternative is not a perfect automated reviewer either. Anthropic's write up on Claude Code's auto mode reports a 17% false negative rate for its own classifier against real overeager actions. Neither a tired human nor a good classifier is a substitute for the agent simply not being able to do the thing. Review is the last line, not the first.
The two questions that decide the list
We put every action an agent can take through two questions, and the answers place it on a grid that tells you what treatment it needs.
Can it be undone, and by whom? Not in principle, in practice, by the person who will be on shift. A database write with a transaction log is reversible. An email to a customer is not. A refund is reversible on paper and expensive in reality.
How far does it reach? One record, one customer, one team, or everyone. An action that touches a single row is a different object from one that touches a table, even when the code path is identical.
| Reversible | Not reversible | |
|---|---|---|
| One record or one customer | Draft a reply. Update a CRM field. Add an internal note to a ticket.Let it run. Log it and move on. | Send that reply to the customer. Issue one refund under a threshold. Post into a shared channel.Let it run, behind a short undo window if you can build one. |
| A segment, a table, everyone | Re-tag four hundred tickets. Reassign a queue. Update a field across a product line.Let it run, but cap the volume per run and alert on the rate. | Email a whole customer segment. Overwrite or delete a table. Change a permission group. Move money above the threshold.Gate it. This is the only cell that always needs a person. |
Notice that draft and send sit in different cells despite being one feature in most people's heads, and that a single refund and a bulk refund are not the same action even when they call the same endpoint. That is usually where the list gets shorter: teams gate a capability when only one of its uses needed gating. Most who map it this way end up with three or four gated actions rather than thirty, and everything else earns a log line, a cap or an alert, none of which cost a person anything.
What makes a gate a control rather than a dialog
Having chosen the actions, the gate itself has to be built so a decision is actually possible. Most are not. A prompt that says the agent would like to run an action, with a name and two buttons, is asking someone to approve something they cannot see.
- Show the actual effect, not the intent. The message body, the amount and recipient, the rows that will change. If the reviewer has to open another system to understand it, they will not.
- Make refusing the default. A gate that times out into approval is a delay wearing a control's clothes.
- Give it a deadline and an owner. An approval nobody is accountable for sits in a queue until someone clears it to make the number go down.
- Record the reasoning, not just the click. When you review an incident later, approved by whom at what time tells you almost nothing. What they were shown tells you everything.
A useful test before you ship a gate: could the person approving it explain, a week later, why they approved that specific one? If the honest answer is no, the gate is generating audit trail rather than safety, and the action underneath it probably needs constraining instead.
The better lever is usually upstream
When the gated list comes out long, the instinct is to hire more reviewers or accept the latency. Nearly always the real fix is upstream: the agent has permissions it does not need for the job it was built to do.
An agent that drafts replies but cannot send them needs no send approval, because it cannot send. An agent that can issue refunds only below a threshold, to accounts it already touched this session, has moved most of its refund traffic out of the gated quadrant without anyone reviewing anything. Narrowing what is possible is cheaper than reviewing what is attempted, and it keeps working at three in the morning when nobody is watching the queue.
This is the same move as the architectural one in the prompt injection piece, applied to a different problem. There, you constrain the agent so a malicious instruction cannot reach a consequential action. Here, you constrain it so an honest mistake cannot either. The mechanism is identical, and one design buys you both.
Where to start
- List every action the agent can take, from the tools it holds rather than from the design document.
- Mark each one reversible or not, in practice, by the person who will be on shift.
- Mark how far each one reaches: one record, one customer, or everyone.
- Gate the wide and irreversible ones. Log, cap or alert on the rest.
- For anything still gated, ask whether a narrower permission would remove the need entirely.
If your gated list is longer than five items, the problem is usually the permissions rather than the review process. Our free AI audit walks through an agent's actual tool permissions and shows which approvals would disappear if the permissions were tightened, which is usually most of them.
Related reading
Get this applied to your business.
The free AI audit measures your live setup and shows where AI would actually pay off.