Token Pricing

Token pricing follows a simple formula, but the details matter for accurate cost calculation:

Request Cost = (Input Tokens × Input Rate) + (Output Tokens × Output Rate)

For a request using GPT-4o with 2,000 input tokens and 500 output tokens:

Cost = (2,000 / 1,000,000 × $2.50) + (500 / 1,000,000 × $10.00)
Cost = $0.005 + $0.005
Cost = $0.01

When prompt caching is active, the formula becomes:

Request Cost = (Cached Input Tokens × Cached Rate) + (Uncached Input Tokens × Input Rate) + (Output Tokens × Output Rate)

For the same request on Claude 3.5 Sonnet with 1,500 cached tokens and 500 uncached tokens:

Cost = (1,500 / 1,000,000 × $0.30) + (500 / 1,000,000 × $3.00) + (500 / 1,000,000 × $15.00)
Cost = $0.00045 + $0.0015 + $0.0075
Cost = $0.00945

Key billing details across providers:

• OpenAI rounds token counts per request before applying pricing. The minimum billable unit is 1 token.
• Anthropic bills exact token counts. Cached tokens require a minimum of 1,024 tokens in the prefix for caching to activate. There is a 25% write premium on the first request that populates the cache.
• Google applies tiered pricing for Gemini 1.5 Pro — requests exceeding 128K tokens pay double the per-token rate.
• All providers include system prompts, tool/function definitions, and message role tags ("user:", "assistant:") in the input token count.

CostHawk normalizes all of these provider-specific billing quirks into a unified cost-per-request metric, so you always know exactly what each request costs regardless of which provider and model you used.

A comprehensive comparison of token pricing across all major providers as of March 2026:

Reading this table for cost optimization:

• The "Cached Input" column shows the effective rate when prompt caching is active. Anthropic's 90% discount on cached input is the most aggressive — if your prompts have consistent prefixes, Anthropic offers the best effective input rate for repeat requests.
• The "Batch Input" column shows rates for asynchronous batch processing. Both OpenAI and Anthropic offer 50% off input tokens for batch requests that tolerate 24-hour latency. Google Gemini offers batch at 50% off as well.
• Mistral does not yet offer prompt caching or batch discounts, but their base rates are very competitive, especially Mistral Small at $0.10/$0.30.

CostHawk maintains an up-to-date pricing database across all providers and automatically applies the correct rates when calculating your costs. When providers change pricing (which happens frequently), CostHawk updates within 24 hours so your cost reports remain accurate.

One of the most important patterns in token pricing is the consistent premium on output tokens. Across all providers, output tokens cost 3–5x more than input tokens:

This ratio has profound implications for cost optimization priorities:

Output-heavy workloads are disproportionately expensive. An application that generates 1,000 output tokens per request pays 4–5x more per output token than per input token. For a 50/50 split of input and output tokens, output accounts for 80% of the total cost.

Optimization priority matrix:

The key insight: for most workloads except pure classification, output token optimization delivers more savings per token reduced than input token optimization. Set max_tokens aggressively, instruct the model to be concise, use structured output formats (JSON) instead of natural language when the consumer is code, and avoid requesting explanations when you only need the answer.

All major providers offer discount mechanisms that can significantly reduce your effective per-token rate. Understanding and leveraging these discounts is one of the highest-ROI cost optimization strategies:

1. Prompt Caching (50–90% input discount)

Prompt caching gives a discount on input tokens when the beginning of your prompt matches a previous request. The mechanics differ by provider:

• Anthropic: 90% discount on cached input tokens. Requires 1,024+ token prefix match. 25% write premium on first request. Cache TTL: 5 minutes (extended with each hit). Best discount in the industry for applications with consistent system prompts.
• OpenAI: 50% discount on cached input tokens. Automatic — no configuration needed. Requires 1,024+ token prefix match. No write premium. Simpler but less aggressive discount.
• Google: Context caching available with explicit API. Pricing varies by model and cache duration. Requires explicit cache creation and management.

Effective savings example: An application with a 4,000-token system prompt, 1,000-token variable context, and 500-token output, making 50,000 requests/day on Claude 3.5 Sonnet:

• Without caching: (5,000 × $3.00/MTok + 500 × $15.00/MTok) × 50,000 = $750 + $375 = $1,125/day
• With caching (90% of system prompt cached): (4,000 × $0.30/MTok + 1,000 × $3.00/MTok + 500 × $15.00/MTok) × 50,000 = $60 + $150 + $375 = $585/day
• Savings: $540/day ($16,200/month)

2. Batch API (50% input discount)

Both OpenAI and Anthropic offer batch processing APIs that accept async workloads with up to 24-hour completion windows. In exchange, you get 50% off input token pricing. Batch is ideal for: data processing pipelines, nightly analysis jobs, content generation workflows, and any workload that does not require real-time response.

3. Volume Commitments

For teams spending $10,000+/month, most providers offer negotiated volume discounts. OpenAI's Committed Use Discount program offers 10–30% off for annual commitments. Anthropic offers custom pricing for enterprise accounts. These discounts stack with prompt caching and batch discounts.

4. Fine-tuned Model Trade-offs

Fine-tuned models charge a premium (2–6x base rate) but can reduce input tokens by eliminating few-shot examples. At high volume, the input savings can outweigh the rate premium. This is a discount in disguise — you pay more per token but use fewer tokens per request.

Accurate cost calculation requires accounting for all token types, discount mechanisms, and provider-specific billing quirks. Here is a comprehensive framework:

Step 1: Identify your token volumes

For each endpoint or feature, measure or estimate:

• Average input tokens per request (including system prompt)
• Average output tokens per request
• Percentage of input tokens that are cacheable (static prefix)
• Daily request volume

Step 2: Apply the correct rates

// TypeScript cost calculator
interface CostParams {
  inputTokens: number
  outputTokens: number
  cachedInputTokens: number
  inputRate: number      // per 1M tokens
  outputRate: number     // per 1M tokens
  cachedRate: number     // per 1M tokens
  requestsPerDay: number
}

function calculateDailyCost(params: CostParams): number {
  const uncachedInput = params.inputTokens - params.cachedInputTokens
  const perRequestCost =
    (params.cachedInputTokens / 1_000_000) * params.cachedRate +
    (uncachedInput / 1_000_000) * params.inputRate +
    (params.outputTokens / 1_000_000) * params.outputRate
  return perRequestCost * params.requestsPerDay
}

// Example: GPT-4o with prompt caching
const daily = calculateDailyCost({
  inputTokens: 3000,
  outputTokens: 500,
  cachedInputTokens: 2000,
  inputRate: 2.50,
  outputRate: 10.00,
  cachedRate: 1.25,
  requestsPerDay: 50000,
})
console.log(`Daily cost: $${daily.toFixed(2)}`)  // $387.50

Step 3: Account for hidden costs

• Retry overhead: If 5% of requests fail and retry, add 5% to your token volume.
• Tool/function definitions: If you use function calling, tool definitions are tokenized and count as input tokens. A set of 10 function definitions can add 500–2,000 tokens per request.
• Reasoning tokens: Models like o1 and o3-mini consume internal thinking tokens that are billed but not visible in the output. These can be 5–20x the visible output token count.
• Conversation history: For multi-turn applications, factor in the growing context size across turns.

Step 4: Build a monthly forecast

Multiply daily cost by 30 and add a 15–25% buffer for usage growth and variance. Update the forecast monthly as actual usage data comes in. CostHawk automates this entire calculation, providing real-time cost tracking, historical trends, and automated forecasting based on your actual usage patterns.

AI token pricing has been declining rapidly and shows no signs of stopping. Understanding the trends helps you plan budgets, evaluate build-vs-buy decisions, and time optimization investments.

Historical price trajectory (OpenAI GPT-4 class models):

In approximately 18 months, GPT-4 class pricing dropped 92% for input and 83% for output. Similar drops occurred across other providers.

Drivers of price decline:

• Hardware improvements: Newer GPU architectures (NVIDIA H100, B100) deliver 2–3x more inference throughput per dollar than predecessors.
• Inference optimization: Techniques like speculative decoding, continuous batching, KV-cache optimization, and quantization allow providers to serve more requests per GPU.
• Competition: With OpenAI, Anthropic, Google, Meta (via open-source), Mistral, and others competing aggressively, pricing pressure is intense.
• Scale economics: As usage grows, providers amortize fixed costs (training, infrastructure) over more tokens, enabling lower per-token prices.

What this means for your budget:

• Costs you commit to today will likely be cheaper in 6 months. Avoid long-term commitments at current rates unless the discount is substantial (30%+).
• Optimize now, benefit twice. Cost optimizations you implement today (prompt caching, model routing, output capping) save money at current prices AND compound with future price cuts.
• Budget for 30–50% annual price decline when forecasting multi-year AI infrastructure costs. This avoids over-provisioning budgets and under-investing in AI capabilities.
• Do not wait for price drops to optimize. The money you save today by optimizing is real money in the bank, regardless of future pricing.

CostHawk tracks pricing changes across all providers and automatically updates cost calculations when rates change. Historical pricing data in CostHawk lets you see how your effective per-token cost has evolved over time and project future trends based on the historical decline curve.