Why track token usage per request in metrics?

Category: Observability & Tracing

Answer

Tokens consumed map directly to cost on most LLM providers. A counter incremented in the LLM call layer lets you plot cost-per-route, alert on a single runaway tenant, and ignore per-call latency variance. Without it you only learn about cost from the invoice.

Concrete examples from the fca project context

Example 1

Counter(“llm_tokens_total”, “Tokens consumed”, [“model”, “route”]).inc(n).

Example 2

Alert if p99 tokens/min for one tenant > 10x median.

Example 3

Tag by model so a price change shows up clearly in the dashboard.

Data flow / flow chart

LLM call -> llm_tokens_total.labels(model="gpt-4o", route="/chat").inc(
  Prometheus -> cost-per-route dashboard

Takeaway

If you measure latency but not tokens, you know speed but not spend. Count both.