AI Agent Cost: The Budget That Decides If It Ships

The agent works. It also takes eleven seconds to answer and costs more per task than the thing it replaced. In the demo nobody noticed. In the release review, finance runs the math to real volume and the project quietly dies. Accuracy got it into the room. Economics decided whether it shipped, and economics said no.

Part 1 called cost and latency a line on the bar. Part 4 is why that line is not a footnote: for an agent, the budget fights everything you did in Part 3 to make it reliable.

Cost is per successful task, not per call

A single model call is cheap. Priced per token, say three cents. But an agent does not make one call. It plans, calls a tool, reads the result, calls another, maybe retries. Eight calls a task is ordinary, so now you are at roughly twenty-four cents a task. At fifty thousand tasks a month that is twelve thousand dollars, for one feature.

Then it gets worse, because not every task succeeds. If the agent succeeds seventy percent of the time and you retry the failures, you pay for about 1.4 attempts per success, so your real number is closer to thirty-four cents per successful task. Cost per call flatters you. Cost per successful outcome is what you actually pay.

Latency compounds, and the tail is what people feel

Those eight calls are also eight round trips, plus tool time. The average task might land in eight seconds, but the distribution has a tail (Part 3), and users feel the tail, not the average. A p95 of twenty seconds means one in twenty people watches a spinner long enough to leave. Latency is not one number. It is a distribution with a cliff.

Reliability and economics pull against each other

Here is the tension nobody puts on the slide. Everything that made the agent more reliable in Part 3, running steps multiple times, adding retries, adding a verification pass, adds cost and latency. Everything that makes it cheaper and faster, fewer steps, a smaller model, no retry, tends to lower reliability. You do not get to maximize both. You design the trade on purpose, against your bar, or the trade designs you in production.

The levers are boring and they work: use a smaller model for the easy steps and the expensive one only where it earns its keep, cache what repeats, cap the number of steps so a task cannot loop forever, fail fast instead of retrying blindly, and set a hard ceiling on cost and latency per task so a runaway is caught by design.

Part 5 closes the series on the last thing between a working agent and a shipped one: what happens when it is wrong, and what stops the damage.

Budget is part of correctness: a right answer you cannot afford never ships.