Turning AI spend into measurable ROI

1. A defensible investment case
   Unit economics in the units your business already reports in, rather than enthusiasm.
2. A cost forecast before you buildWhat a feature will cost at expected volume, instead of what it cost once the invoice arrived.
3. A list of what deserves more moneyThe surfaces returning several times their cost. This is the move nobody makes today, because nobody can prove which ones they are.

Your AI bill contains two different spends, and they need to be split apart in order to get a true picture of where the money is going.

Build spend

What your engineers consume while building: coding agents, chat, review tooling. Its dominant waste is rework, and the team generating the cost is the team that can fix it.

Run spend

What your product consumes while running: the agent answering a customer, the pipeline classifying a document, the enrichment job nobody has looked at since launch. It scales with usage, so it grows when the business grows.

Tag spend at the source. Set up provider billing at project or key level, plus a naming convention that someone owns and new surfaces inherit automatically. It takes a few days and is the cheapest stage in the process.

From here, we follow run spend: the agentic systems in production. Build spend is a separate problem with its own data.

Cost per token tells you nothing a business can act on. Cost per outcome does.

For every AI surface, name the business outcome it supports and define its unit: cost per inquiry resolved, per document analyzed, per transaction processed, per candidate screened.

Then instrument per-feature attribution against that unit. You need request-level tagging tied to a feature identifier, and the outcome count from whichever system already records it. Most organizations have both halves and have never joined them.

Pick the unit your finance team already uses. If support reports cost per resolved ticket, your AI number should be cost per resolved ticket. The point is not precision for its own sake; it is that the figure lands in a budget conversation without translation.

Cost per outcome shows the spend was efficient. The next step is to trace that outcome into a number the business already tracks outside the AI budget. Check whether resolved inquiries show up in CSAT or in mean time to resolution. An efficient number that nobody benefits from is still a failure, and so is a benefit nobody can afford. Efficiency and business outcome together prove the spend delivered value.

Record your eval baseline before you change anything.

Record what working looks like. That record is your eval baseline, and it is what turns “we made this cheaper” into “we made this cheaper and here is the evidence quality held.” Without it, every proposed change is a bet nobody is willing to authorize.

Start with the two cheap tiers. Programmatic evals check observable signals: required structure present, forbidden content absent. Performance-based evals analyze outcomes in production: did it work for the user, did it move the business. Both scale cheaply. Add judged evaluation, where a model scores behavioral compliance against explicit validation points, only where behavior matters more than structure. Reserve expert panels for the surfaces where a domain expert’s read is the only real test.

Then make it automatic. Treat every model provider update, context change, and tool interface change as a deployment event that runs the suite and compares against the recorded baseline.

Someone has to own the check itself, periodically confirming that resolved still means resolved as volume grows, rather than trusting the count on faith.

Teams tend to skip this stage, and that’s why the cost data they collected in Stage 2 never gets acted on. The baseline is also the measurement that shows the system is still safe to run unsupervised. Because you can rerun it, you can keep trusting a system with autonomy as it changes underneath you.

Now the numbers can make decisions, and they point in three directions rather than one.

Fund: expand what’s working

Some surfaces return several times what they cost. Give them more volume and coverage, or move them to a premium model where a cheaper one is capping the outcome. This call is invisible without Stages 2 and 3, and it’s usually worth more than anything in the other two categories.

Fix: improve what’s inefficient

Some surfaces produce real value at a higher cost than they should. Route routine work to affordable models and send complex cases to premium ones, with automatic escalation when a case needs more. Tightening prompts and context and optimizing for cache hits trim the cost further, and a single call can often replace a multi-turn flow.

Stop: retire what doesn’t pay

Some surfaces cost more than they return, or return something nobody downstream uses. Retiring one creates a category of savings that no amount of routing can reach, and it’s only available when you can prove the case.

Autonomy limits

Every decision runs inside a limit on what an agent may do on its own, set by data sensitivity and reversibility. Funding more autonomy is safe only where the ceiling and the quality baseline both allow it. Governance enforces that ceiling at execution time.

Everything so far has been about spending what you already have. Now, we’re talking about spend you haven’t committed yet, and it is where the work starts paying finance back rather than engineering.

Once you have cost per outcome for existing surfaces, you can estimate a new feature before it is built. Take the closest comparable surface, adjust for the differences you can name, multiply by the volume the product team expects.

The estimate won’t be exact, and its margin of error narrows every quarter as your comparables improve. It still beats the current method of finding out in arrears.

To change decisions, the estimate has to sit in the review gate alongside the security review and the architecture sign-off. The question it answers is whether the unit economics still work at ten times the pilot volume.

A good forecast tells you early, while changing your mind is still cheap, which features need a different design to be worth building.

Nothing here stays accurate on its own. Models change, volume grows, new surfaces inherit none of it. Six practices keep it current:

1. Cost-per-outcome dashboards
   At the feature and system level. Backed by two review gates: the Stage 5 estimate before build, the actual number before an agentic workload scales.
2. Alerts tied to business metrics
   Not raw spend. A 20% cost increase on something whose volume just doubled shouldn’t page anyone.
3. A named owner
   One owner per agentic workload, who sees cost, outcome, and quality. An agentic workload will get cheaper while quietly getting worse if a qualified and accountable person is not watching all three. Each measure needs a single definition, so in the case of the support agent, the owner also signs off on the definition of “resolved”.
4. Cost limits enforced in code
   An agent with no mechanical ceiling has no spending limit in the time between a change of behavior and the next accounting cycle.
5. Autonomy limits enforced in code
   Scaled to two things: how sensitive the target is and whether the action can be undone. A budget cap won’t stop an irreversible action taken within budget.
6. A standing process
   For evaluating new model tiers as providers release them, since the cheapest correct model changes several times a year.Document all of this so new engineers and successor owners inherit the cost model from day one and the practice outlasts the people who built it.

Check each statement that’s true today, not planned. Each stage’s score is the number checked, out of three. Start at the lowest-numbered stage scoring below three.