Your engineering team’s AI bill went from $500 to $5,000 in a month. Nobody knows why. Nobody’s accountable. This is the AI cost governance problem — and it’s hitting every team that scales beyond a few developers.

The problem

AI costs are invisible in traditional budgeting:

• No server to point at (“that EC2 instance costs $X”)
• Costs scale with usage, not headcount
• Individual developers make model choices that affect the bill
• No approval workflow for “I’ll use Claude Opus instead of Sonnet”

The framework

Level 1: Visibility (week 1)

Before you can control costs, you need to see them.

Track per-request costs:

cost = (input_tokens * input_price + output_tokens * output_price) / 1_000_000
logger.info({"team": team, "feature": feature, "model": model, "cost": cost})

Weekly cost report:

Total: $4,200
  Backend team: $2,100 (50%)
    Code review: $1,400
    Test generation: $500
    Documentation: $200
  Frontend team: $1,200 (29%)
  Data team: $900 (21%)

Use Helicone for automatic tracking or build a simple middleware.

Level 2: Budgets (week 2)

Set limits at three levels:

Level	Budget	Who sets it	Who’s accountable
Company	$X/month	Finance	CTO
Team	$X/month	Engineering manager	Team lead
Feature	$X/month	Tech lead	Feature owner

Alert thresholds:

• 50%: Informational (Slack message)
• 75%: Warning (email to team lead)
• 90%: Urgent (email to manager + Slack)
• 100%: Hard stop or approval required

See our monitoring guide for implementation.

Level 3: Optimization (week 3+)

Once you see where money goes, optimize:

1. Model routing — Are developers using Opus for tasks Sonnet handles? Route automatically.

2. Prompt caching — Are the same system prompts sent repeatedly? Cache them.

3. Token budgets per request — Set max_tokens to prevent runaway generation.

4. Self-hosting for predictable workloads — If a feature uses 10M tokens/month of the same model, self-host it.

Level 4: Accountability (ongoing)

Make AI costs visible in the same way cloud costs are:

• Monthly cost review — 30-minute meeting reviewing AI spend by team/feature
• Cost per PR — Show AI cost alongside CI/CD time in pull requests
• Model choice justification — If a developer uses Opus, they should know it costs 50x more than DeepSeek

The cultural shift

The goal isn’t to minimize AI spending — it’s to spend wisely. A $500 Claude Opus session that finds a critical bug is worth it. A $500 session that reformats CSS is not.

Engineers need to think about AI costs the same way they think about database queries: use the right tool for the job, don’t over-provision, and monitor what you use.

Common cost traps

The “just use Opus for everything” trap

Developers default to the best model because it’s easiest. But 80% of tasks don’t need frontier models:

Task	Right model	Wrong model	Cost difference
Code autocomplete	Codestral	Claude Opus	50x cheaper
Simple refactoring	Claude Haiku	Claude Opus	30x cheaper
Architecture review	Claude Opus	Claude Opus	Correct choice
Test generation	DeepSeek	Claude Opus	100x cheaper

The “no caching” trap

If 100 developers on your team use the same system prompt (2,000 tokens), that’s 200,000 wasted tokens per day. Prompt caching reduces this to near zero.

The “unlimited context” trap

Developers paste entire codebases into context windows because they can. A 200K token context costs 10-50x more than a focused 10K token context. Teach teams to be selective about what they include.

Implementation timeline

Week	Action	Effort
1	Add per-request cost logging	2 hours
2	Set up weekly cost report	1 hour
3	Set budget alerts at 75%/90%	1 hour
4	Implement model routing for common tasks	4 hours
Ongoing	Monthly cost review meeting	30 min/month

Total setup: one day of engineering work. The ROI is typically 30-50% cost reduction within the first month.

Tools

Need	Tool
Cost tracking	Helicone (proxy, automatic)
Unified billing	OpenRouter (one bill for all models)
Budget enforcement	Custom middleware or Portkey
Reporting	Helicone dashboard or custom Grafana

See our FinOps for AI guide for the full enterprise framework.

Related: FinOps for AI · How to Reduce LLM API Costs · Monitor and Control AI Spending · LLM Cost Calculator