AI agents burn money fast. A single autonomous coding session can cost $5-50 in API calls. Multiply that across a team of agents running 24/7, and costs spiral without monitoring.
Here are the tools that track agent API spend, alert on budget overruns, and help you optimize costs.
Why agent cost monitoring matters
Agents are not like regular API calls. They:
- Run for hours or days (not milliseconds)
- Make hundreds of sequential API calls per task
- Retry on failures (costing more)
- Can loop indefinitely (budget disasters)
- Use different models for different steps (mixed pricing)
Without monitoring, you won’t know your agent burned $200 until the bill arrives.
The tools
1. Helicone
Best for: Developers who want a quick setup.
| Feature | Details |
|---|---|
| Setup | One-line proxy change |
| Models | All major providers |
| Cost tracking | Per-request, per-agent, per-user |
| Alerts | Budget limits, anomaly detection |
| Pricing | Free tier, $20/month pro |
Helicone works as a proxy. Change your API base URL to Helicone’s, and it logs every request with cost data. No code changes needed.
Strengths:
- Easiest setup (one line change)
- Per-request cost tracking
- Budget alerts
- User-level cost attribution
Weaknesses:
- Proxy adds latency (~50ms)
- Limited agent-specific features
- No built-in cost optimization
2. LangSmith
Best for: Teams using LangChain or LangGraph.
| Feature | Details |
|---|---|
| Setup | LangChain integration |
| Models | All major providers |
| Cost tracking | Per-chain, per-agent, per-step |
| Alerts | Budget limits |
| Pricing | Free tier, $39/month developer |
LangSmith is the observability platform for LangChain. If you’re building agents with LangChain or LangGraph, LangSmith gives you detailed traces with cost data at every step.
Strengths:
- Deep LangChain integration
- Step-level cost visibility
- Trace visualization
- Agent debugging tools
Weaknesses:
- LangChain lock-in
- More expensive than alternatives
- Overkill for non-LangChain agents
3. Portkey
Best for: Multi-provider agent setups.
| Feature | Details |
|---|---|
| Setup | SDK or proxy |
| Models | 200+ models, all providers |
| Cost tracking | Per-model, per-agent, per-route |
| Alerts | Budget limits, fallback routing |
| Pricing | Free tier, $49/month team |
Portkey is an AI gateway. It routes requests to different models based on cost, latency, or quality. Built-in cost tracking across all providers.
Strengths:
- Multi-provider cost aggregation
- Intelligent routing (cost/quality tradeoffs)
- Fallback handling
- Budget management
Weaknesses:
- More complex setup
- Gateway adds a dependency
- Pricing is higher
4. Langfuse
Best for: Self-hosted observability.
| Feature | Details |
|---|---|
| Setup | Self-hosted or cloud |
| Models | All major providers |
| Cost tracking | Per-trace, per-generation |
| Alerts | Via webhooks |
| Pricing | Free (self-hosted), $50/month cloud |
Langfuse is open-source and can be self-hosted. Good for teams that need data privacy or want to customize their observability stack.
Strengths:
- Open source, self-hostable
- Detailed trace visualization
- Cost per trace/generation
- No data leaves your infra (self-hosted)
Weaknesses:
- Self-hosting requires maintenance
- Less polished than commercial options
- Smaller community
5. Custom dashboards
For teams that need full control.
Build your own with:
- Prometheus + Grafana: For metrics and dashboards
- ClickHouse: For high-volume log analysis
- PostgreSQL: For simple cost tracking
# Example: Log agent costs to PostgreSQL
import psycopg2
def log_agent_cost(agent_id, model, input_tokens, output_tokens, cost):
conn = psycopg2.connect("postgresql://localhost/agent_costs")
cur = conn.cursor()
cur.execute("""
INSERT INTO agent_costs (agent_id, model, input_tokens, output_tokens, cost, timestamp)
VALUES (%s, %s, %s, %s, %s, NOW())
""", (agent_id, model, input_tokens, output_tokens, cost))
conn.commit()Comparison
| Tool | Setup | Agent-Specific | Self-Hosted | Free Tier | Price |
|---|---|---|---|---|---|
| Helicone | Easiest | Limited | No | Yes | $20/mo |
| LangSmith | LangChain | Yes | No | Yes | $39/mo |
| Portkey | Moderate | Yes | No | Yes | $49/mo |
| Langfuse | Moderate | Yes | Yes | Yes | $50/mo |
| Custom | Hard | Full control | Yes | N/A | Varies |
What to monitor
Regardless of which tool you choose, track these metrics:
1. Cost per agent task: How much does each autonomous task cost?
2. Cost per model: Which model is consuming the most budget?
3. Retry costs: How much is spent on retries and error recovery?
4. Token efficiency: Are agents using more tokens than necessary?
5. Budget burn rate: How fast are you approaching budget limits?
6. Cost per outcome: What’s the cost per completed task, per PR merged, per issue resolved?
My take
For most developers, Helicone is the right starting point. One-line setup, free tier, and per-request cost tracking. It’s not agent-specific, but it gives you the data you need.
For LangChain teams, LangSmith is the obvious choice. The deep integration gives you step-level visibility that generic tools can’t match.
For teams that need self-hosting, Langfuse is the best option. Open-source, self-hostable, and good trace visualization.
For production agent deployments, build a custom dashboard alongside one of these tools. The commercial tools are great for development, but production needs custom metrics (cost per outcome, budget burn rate, agent efficiency).
FAQ
How much do agent cost monitoring tools cost?
Free tiers are available for all major tools. Paid plans start at $20/month (Helicone) to $50/month (Langfuse cloud). Self-hosted Langfuse is free.
Can I track costs without a monitoring tool?
Yes, but you need to log every API call with token counts and calculate costs manually. Most providers (OpenAI, Anthropic) have dashboards, but they don’t aggregate across providers or attribute costs to specific agents.
What’s the most important metric to track?
Cost per completed task. Total spend is meaningless without knowing what you got for it. Track cost per PR merged, per issue resolved, per document processed.
How do I set budget alerts?
Most tools support budget limits. Set daily/weekly/monthly limits and configure alerts at 50%, 80%, and 100% of budget. For autonomous agents, set per-task limits too.
Should I use a proxy or SDK?
Proxy is easier to set up (one line change). SDK gives more control but requires code changes. Start with proxy, switch to SDK if you need agent-specific features.