LLM spend: attribution before optimisation
You cannot cut what you cannot see. Most teams optimise models and prompts before they can attribute a single dollar to a team, feature or customer, which means they are guessing. Meter at the gateway first, then cache, route and cap in that order.
The conversation about AI spend nearly always starts in the wrong place. Someone notices the invoice, and the response is to switch to a cheaper model, trim prompts, or add a cache. All reasonable, all premature, because almost nobody in the room can say which feature spent the money.
You cannot cut what you cannot see. Optimising before you can attribute is guessing with extra steps, and it tends to produce a quarter of work that makes the number move slightly while nobody can explain why.
The scale is real. CloudZero's State of AI Costs report, from a survey of 500 US software engineers and senior managers at firms between 250 and 10,000 employees conducted in March 2025, states that "in 2024, the average monthly spend on AI was $62,964, which our findings suggest will rise to $85,521 in 2025 (a 36% increase)". Note that the second figure is the report's projection rather than a measurement. The more telling finding sits beside it: only around half of organisations strongly agreed they could track AI return effectively, while nine in ten expressed general confidence. That gap between feeling able to evaluate spend and actually being able to is the problem.
Attribution first, and it has to be at the gateway
Attribution means every model call arrives at your provider carrying enough metadata to answer who spent this and on what. Doing it in application code means it is done inconsistently, and the calls nobody instrumented are exactly the ones you need.
Route everything through one service you own and tag at that boundary. Four dimensions cover almost every question worth asking.
- Team or service, so cost lands with whoever can change it.
- Feature, because "support assistant" and "internal search" have different economics and different owners.
- Customer or tenant, which is how you discover that four percent of accounts drive forty percent of inference.
- Environment, because a surprising share of spend is staging and nobody looks.
Log tokens in and out per call alongside those tags. Provider invoices are monthly, coarse and arrive too late to change anything; your own record is queryable today and lets you answer "what changed on Tuesday" while Tuesday still matters.
The agentic failure mode
Everything above is ordinary engineering discipline. Agents introduce a genuinely different risk, which is that spend is no longer proportional to usage.
A chat feature costs roughly what its traffic implies. An agent decides how much work to do. A retry loop that fails a validation and tries again, a plan that expands into more subtasks than expected, a tool returning an error the agent responds to by trying harder: any of these can consume in minutes what the feature was budgeted for in a month, at three in the morning, with no user waiting for the result.
Rate limits do not save you here, because the traffic is not anomalous. One legitimate request is doing an unbounded amount of work. The controls that hold are a hard cap on spend per task, a maximum step count, and a maximum depth if agents can call agents, all enforced where the call is made rather than checked afterwards.
We cover the related constraint in agent to agent interoperability: the moment agents can delegate, the cost of a task stops being something you can read off the top level request.
Then optimise, in this order
Once you can attribute, the order matters, because each step is more disruptive than the one before it.
- Cache. Identical and near identical requests are more common than teams expect, particularly in internal tools. Cheapest possible win, no behaviour change.
- Trim context. Most prompts accumulate material nobody has re-read since it was added. Measure tokens per call by feature and the outliers will be obvious.
- Route by difficulty. Send easy work to a smaller model and hard work to a large one. This needs a real evaluation set to do safely, which is why it comes after the free wins.
- Cap. Per task, per feature, per day. Caps are not cost control, they are blast radius control, and they should be set even when spend is comfortable.
Notice that switching provider, the thing usually proposed first, does not appear. It is the largest disruption for a discount that is frequently smaller than what caching and context trimming return, and it is a decision worth making on capability rather than price.
The number worth knowing per customer
Once attribution works, one derived figure is worth more than the total: inference cost per customer, per month, next to what that customer pays.
Aggregate spend hides the distribution, and the distribution is where the decisions are. A handful of accounts using the product exactly as intended can consume more than they pay while the average looks comfortable. Nobody notices until growth makes it structural, at which point the fix is a pricing change rather than an engineering one, and pricing changes take a quarter.
This matters more for AI features than for ordinary software because the marginal cost of a heavy user is real rather than rounding. Compute the ratio, sort descending, and look at the top ten. That view has changed more roadmaps than any optimisation we have shipped.
The check worth running this week
Pick last month's AI invoice and try to divide it across your features, from your own records rather than estimates. If you cannot get within a rough margin, that is the project, and it is a week of work rather than a quarter. Everything else people are proposing depends on it.
If you want the gateway and the tagging designed once so this stays answerable as you add features, that is what our AI gateway work covers, and a free AI audit will tell you how much of your current spend is currently attributable.
Related reading
Get this applied to your business.
The free AI audit measures your live setup and shows where AI would actually pay off.