Agent Cost Control: Budgets, Caching, and Model Routing

Agents fail economically when cost per successful task exceeds human or workflow alternatives. Control spend without blindly crippling quality.

Measure the right unit

Track:

• Cost per run
• Cost per successful task (primary)
• Tokens by node (plan/act/critique)
• Tool $ and third-party API $
• Human review $ for HITL steps

A cheaper agent with half the success rate can be more expensive.

Levers that work

1. Hard budgets

Max steps, max tokens, max wall-clock, max $. Enforce in the runtime—not the prompt.

2. Model routing

• Strong model for planning / hard judgments
• Smaller model for extraction, classification, formatting
• Deterministic code for schema transforms

3. Better tools, fewer steps

High-quality tools reduce thrashing. A precise get_order(id) beats five fuzzy searches.

4. Context hygiene

• Summarize trajectories
• Don’t paste entire HTML/PDFs
• Cache stable system prompts and retrieved docs

5. Stop policies

Exit when success criteria are met; don’t “polish forever.”

6. Multi-agent discipline

Each agent message has a cost. Require clear handoff contracts; avoid debate clubs in production.

7. Caching and idempotency

Cache search results and deterministic transforms within a run; dedupe identical tool calls.

Design patterns for thrift

Pattern	Savings mechanism
Plan-then-execute	Fewer aimless tool calls
Router agent	Send only hard cases to heavy models
Critic only on risk	Skip expensive review for low-impact outputs
Workflow backbone	Agents only for exceptions

Alerts and governance

• Budget burn alerts mid-run
• Daily spend caps per tenant
• Kill runaway jobs
• Weekly cost/success scoreboard by intent

Eval with economics

In evaluation, report:

success_rate
median_cost_success
p95_cost_success
cost_at_target_quality

Optimize for a target success rate at minimum cost—not minimum cost alone.

Summary

Cost control is a runtime and product problem: budgets, routing, tools, and stop conditions. Measure success-adjusted cost or you will “save money” by shipping an agent that fails.