Skip to Content

Tokens, Context Window and Cost Budgets

When to use

Plan around the model context window and the per-1M-token price — the two non-negotiable constraints of every production prompt.

Analogy

A workspace whiteboard: when you exceed it the oldest information gets erased. Pricing is per square inch of whiteboard you actually use.

Data-flow diagram

   model         context    $/M input   $/M output
   gpt-4o-mini    128 k       0.15        0.60
   gpt-4o         128 k       2.50       10.00
   claude-haiku   200 k       0.25        1.25

Deep explanation

Tokens are the atomic unit: a word is roughly 1.3 tokens in English. Each model has a maximum context. Pricing is tiered: input tokens are cheaper than output. Engineering the budget: keep system prompt tiny; pre-compute reference material; cache static context; avoid round-trips that re-send history.

Examples

Example 1

import tiktoken
enc = tiktoken.encoding_for_model('gpt-4o-mini')
print('tokens:', len(enc.encode('Prompt engineering is fun.')))

tiktoken counts tokens in real time without sending the prompt.

Example 2

def cost(p_tokens, c_tokens, in_per_m=0.15, out_per_m=0.6):
    return (p_tokens * in_per_m + c_tokens * out_per_m) / 1_000_000
print('cost per call:', cost(2000, 500))

Cost calc is the standard sanity check before shipping — most teams catch overspend here.

Example 3

def trim_to_budget(messages, max_tokens=4000):
    enc = tiktoken.encoding_for_model('gpt-4o-mini')
    out, used = [], 0
    for m in reversed(messages):
        t = len(enc.encode(m['content']))
        if used + t > max_tokens: break
        out.insert(0, m); used += t
    return out

Trim-to-budget lets long conversations survive past the context window.

Common mistake

Sending the whole conversation history on every turn without counting. The prompt balloons past the budget, costs spike, model forgets early context.

Key takeaway

Count tokens before sending. Trim oldest turns when over budget. Output is more expensive than input. Monitor spend per request type.

Production Failure Playbook

Failure scenario 1: context-overflow-truncation

Failure scenario 2: output-spike