AI agent vs chatbot: which does your business actually need?

A chatbot answers. An agent acts. If resolving the request means changing something in one of your systems, you need an agent. If the answer already exists in a document, you do not, and buying one costs more and carries more risk.

JarvisBitz

Almost every vendor now calls their product an agent. Most of them are selling a chatbot with better marketing. The distinction matters because the two things have different costs, different risks, and different failure modes, and buying the wrong one is expensive in both directions.

The difference is action, not intelligence

Both use the same underlying models. Both can hold a conversation. The line between them is simple:

• A chatbot produces text. It reads your documents, retrieves relevant information, and composes an answer. Everything it does ends in words on a screen.
• An agent produces changes. It decides on a sequence of steps, calls tools that touch real systems, checks whether the result worked, and retries or escalates when it did not.

A chatbot can tell a customer your refund policy. An agent can issue the refund. That gap is where most of the engineering, most of the cost, and all of the risk lives.

What each is genuinely good at

How to tell a real agent from a relabelled chatbot

Vendor demos are designed to blur this. Four questions cut through it, and you can ask them in a sales call:

• What can it change without a human? If the honest answer is nothing, it is a chatbot. That is not a criticism, but you should not pay agent prices for it.
• What tools can it call, and who authorised them? A real agent has a defined tool list with permissions. Vagueness here usually means the actions are hardcoded flows, which is automation with a chat interface.
• What happens when a step fails halfway through? Agents need to detect failure, retry, and roll back. If the answer is that it apologises to the user, nothing was actually transactional.
• Show me a run trace. Any production agent logs each step, tool call and decision. If they cannot show you one, it has not run in production.

The decision test

Take your ten most common requests and sort them into two piles. In the first, the resolution is information the person did not have. In the second, resolution requires something in a system to change.

If nearly everything lands in the first pile, build the retrieval system and stop. It is cheaper, ships sooner, and carries almost no blast radius when it is wrong. If a meaningful share lands in the second pile, and handling those manually is what actually costs you, that is where an agent earns its cost.

Most businesses we talk to have more in the first pile than they expect, which is why we often recommend the cheaper option.

Getting it wrong is not symmetrical

Both mistakes are recoverable and they do not cost the same, which is worth knowing before you make the call under uncertainty.

Build a chatbot where an agent was needed and the result is disappointing but contained. People get an answer and then do the work themselves, exactly as they did before. You have underdelivered, the gap is obvious within a week, and the fix is to extend the thing you already have.

Build an agent where a chatbot would have done and the costs are less visible and harder to reverse. You have given a system permissions it did not need, which is now a surface someone has to review and defend. You are paying for tool calls and retries on work that was a lookup. And you have taken on a class of failure that did not previously exist, where the system does something wrong rather than says something wrong, and somebody has to find it and undo it.

The asymmetry is the practical argument for starting narrower than feels ambitious. If the honest answer is that you are not sure, build the thing that answers, watch where people take over manually, and let that evidence tell you which actions are worth the permissions.

What an agent needs that a chatbot does not

If you do need one, budget for the parts that are not the model. This is the work that separates a demo from something you can leave running:

• Scoped permissions so it can only touch what it should, per user and per action.
• Human approval on high impact steps, with a sensible threshold rather than approving everything.
• Idempotent writes, so a retry does not create a second refund or a duplicate order.
• A full run trace for every task, because you will need to explain what happened.
• Evaluations that catch quality regressions before your customers do.

None of that is exotic, but all of it is real engineering, and it is the honest reason agents cost more.

One thing this page deliberately does not settle: having decided you need something that acts, you still have to decide whether your code or the model chooses each next step. That is a separate question with its own answer, in most agents should be workflows.

The pattern that usually works

Start with retrieval, and let it answer the questions that have answers. Instrument it, so you can see which requests it cannot resolve and why. Then add actions to the specific cases where the data shows a human is doing repetitive work, one action at a time, each with its own guardrails.

That sequence gives you value in weeks rather than quarters, and by the time you build the agent you are automating a workflow you now have evidence about, rather than one you assumed.

If you want to see which of your workflows fall on each side of that line, our free AI audit looks at your live setup and says where automation would actually pay off. Or read how we build retrieval systems if the first pile is where your volume sits.

Related reading

A2A explained: when AI agents need to talk to each other

Most businesses do not need it yet. A2A standardises how agents built by different teams or vendors delegate work to each other, which only starts to matter once you have more than one agent and the other one is not yours to change.

Prompt injection: why filters fail and what actually works

Not with filters. A model cannot tell where an instruction came from, so any defence that tries to spot a malicious one will eventually be talked past. What holds is constraining what the agent is able to do after it reads untrusted content, which is an architecture decision rather than a prompt.

Where human approval actually belongs in an AI agent

In front of actions that cannot be undone, or that reach further than a blast radius you have deliberately chosen. Gating everything is close to gating nothing, because a control that fires constantly stops being read. The design work is deciding which actions qualify, then shrinking permissions so the list stays short.