Unit Economics

AI unit economics is the discipline of measuring the cost and revenue of a single atomic interaction in your AI-powered product. The concept borrows from traditional unit economics — a framework used in e-commerce (cost per order), ride-sharing (cost per ride), and SaaS (cost per customer) — but applies it to the unique cost structure of AI inference workloads.

In traditional software, serving an additional request costs nearly nothing. The server is already running, the code is already deployed, and the marginal cost of one more database query is measured in fractions of a cent. AI changes this equation fundamentally. Every LLM API call incurs a real, measurable cost that scales linearly with usage. There is no amortization, no economy of scale on the inference itself — 1,000 queries cost roughly 1,000 times what one query costs.

This makes AI products behave more like services businesses than software businesses from a cost perspective. A consulting firm incurs labor costs for every client engagement. An AI product incurs inference costs for every user interaction. The parallel is direct, and it demands the same rigor in tracking per-unit profitability.

AI unit economics encompass several cost components beyond the obvious API call:

• Direct inference cost: The per-token charges from your LLM provider (OpenAI, Anthropic, Google, etc.). This is typically 60-80% of total per-unit cost.
• Embedding and retrieval cost: If your product uses RAG (retrieval-augmented generation), every query may trigger embedding generation ($0.02-$0.13 per million tokens) and vector database queries ($0.001-$0.01 per query depending on your vector DB provider).
• Orchestration overhead: Multi-step agent workflows that make 3-8 LLM calls per user query multiply the direct inference cost proportionally. A coding assistant that plans, writes, reviews, and revises code may make 5 API calls to serve a single user request.
• Pre/post-processing compute: Document parsing, image processing, audio transcription, and output formatting all consume compute resources that contribute to per-unit cost.
• Infrastructure allocation: Server costs, database queries, logging, and monitoring attributable to each unit. While smaller than inference costs, these add 10-20% to the total per-unit cost at scale.

A complete unit economics model accounts for all of these components, not just the headline API cost. CostHawk tracks the direct inference cost automatically through wrapped keys and MCP telemetry. By combining this with your infrastructure cost data, you can build a comprehensive per-unit cost model that reflects the true economics of your product.

The most critical decision in AI unit economics is choosing the right unit of analysis. The wrong choice produces misleading metrics that drive poor business decisions. The right choice gives you a clear, actionable view of profitability at the level where you can actually influence it.

There are four common unit definitions for AI products, each appropriate for different business models:

1. Per Query / Per Request

The most granular unit. One query equals one user interaction that produces one AI response. This is the right unit for products where each interaction is independent and self-contained: search engines, classification tools, single-turn Q&A systems, and content generation tools where each generation is a separate deliverable.

Per-query economics are straightforward to calculate: take the API cost of that single request (input tokens + output tokens priced at model rates) and add the proportional infrastructure cost. Revenue attribution is the challenge — if users pay a flat subscription, you must allocate a portion of their monthly payment to each query.

Typical per-query costs in production (March 2026):

• Simple classification (GPT-4o mini): $0.0001-$0.001
• Short-form generation (GPT-4o): $0.003-$0.02
• Long-form generation (Claude 3.5 Sonnet): $0.01-$0.15
• Complex reasoning with tools (GPT-4o + function calling): $0.02-$0.30
• Multi-step agent workflow (3-8 calls): $0.05-$1.20

2. Per Session / Per Conversation

A session groups multiple queries into a single user engagement. This is the right unit for chatbots, coding assistants, and any product where users have multi-turn interactions. Session economics capture the full cost of an engagement, including the growing context window and escalating per-turn costs that individual query metrics miss.

A typical chatbot session might span 8-15 turns. Due to conversation history accumulation, later turns are dramatically more expensive than earlier ones. Turn 1 might cost $0.005, while turn 15 (with 14 turns of history in the context) might cost $0.08. Per-session analysis reveals this compounding dynamic that per-query averages obscure.

3. Per User / Per Customer

The classic SaaS unit. Revenue attribution is simple (the subscription price), and costs aggregate all AI usage by that user over a billing period. This unit is ideal for identifying profitable versus unprofitable customer segments and for setting pricing tiers. The danger is over-aggregation: a per-user view may hide the fact that one feature is profitable while another is deeply loss-making.

4. Per Transaction / Per Output

For products that deliver a discrete, valuable output — a generated report, a processed document, a completed code review, an analyzed image batch — the transaction is the natural unit. This maps well to usage-based pricing models where you charge per document processed or per report generated.

CostHawk supports all four unit definitions through its tagging system. Tag requests with user_id, session_id, feature, or transaction_id to aggregate costs at whatever level matches your business model. Most mature AI products track unit economics at multiple levels simultaneously — per query for optimization, per session for product analytics, and per user for pricing strategy.

Calculating AI unit economics requires combining data from multiple sources into a single per-unit cost figure. Here is the step-by-step methodology used by teams with mature AI cost practices:

Step 1: Measure direct inference cost per unit

This is the sum of all LLM API charges attributable to one unit. For a single-call unit, it is simply:

inference_cost = (input_tokens / 1,000,000) * input_price + (output_tokens / 1,000,000) * output_price

For a multi-call unit (agent workflows, multi-turn sessions), sum across all calls:

total_inference_cost = SUM(
  (call.input_tokens / 1M) * call.model_input_price +
  (call.output_tokens / 1M) * call.model_output_price
  for each call in unit
)

CostHawk calculates this automatically for every request routed through wrapped keys. The dashboard shows per-request cost with model-level breakdowns, and you can aggregate by any tag to get per-session, per-user, or per-feature totals.

Step 2: Add embedding and retrieval costs

If your product uses RAG, each unit incurs embedding generation costs and vector database query costs:

rag_cost = embedding_cost + vector_query_cost

// Example: text-embedding-3-small at $0.02/1M tokens
// Query text: ~200 tokens = $0.000004
// Vector DB query (Pinecone): ~$0.008 per query
// Total RAG overhead: ~$0.008 per unit

For products with heavy RAG usage, this can add $0.005-$0.02 per unit, which is significant at scale.

Step 3: Account for orchestration multipliers

Agent frameworks (LangChain, CrewAI, AutoGen) often make multiple LLM calls per user-visible interaction. A "simple" code review might involve: (1) analyze the diff, (2) identify issues, (3) generate suggestions, (4) format output — four LLM calls. Track the average number of LLM calls per user-visible unit:

orchestration_multiplier = avg_llm_calls_per_unit
effective_inference_cost = avg_single_call_cost * orchestration_multiplier

Many teams are shocked to discover their orchestration multiplier is 4-8x, meaning each user interaction costs 4-8 times what a single API call would suggest.

Step 4: Allocate infrastructure costs

Distribute your monthly infrastructure bill (servers, databases, monitoring, logging) proportionally across units:

infra_cost_per_unit = monthly_infra_cost / monthly_unit_volume

// Example: $2,000/month infra, 500,000 queries/month
// Infra per unit: $0.004

Step 5: Calculate total cost per unit

total_cost_per_unit = inference_cost + rag_cost + infra_cost_per_unit

// Example:
// Inference: $0.045
// RAG: $0.008
// Infra: $0.004
// Total: $0.057 per query

Step 6: Calculate revenue per unit

revenue_per_unit = monthly_revenue / monthly_unit_volume

// Example: $49 subscription, 500 queries/month
// Revenue per query: $0.098

Step 7: Determine unit margin

unit_margin = revenue_per_unit - total_cost_per_unit
unit_margin_percent = (unit_margin / revenue_per_unit) * 100

// Example: $0.098 - $0.057 = $0.041 margin (41.8%)

A healthy AI product targets 40-60% unit margins after inference costs to leave room for sales, support, and R&D. Below 30% is a warning sign. Below 0% means you are losing money on every interaction and growth accelerates your losses.

Unit economics vary dramatically across AI product categories. The table below shows typical ranges for common AI product types based on aggregated industry data and CostHawk customer benchmarks as of March 2026:

Several patterns emerge from this data:

Document analysis has the best unit economics because the output is high-value (legal analysis, medical summaries) and customers expect to pay meaningful per-document fees. The cost is higher than simpler use cases, but the revenue-to-cost ratio is favorable.

AI agents have the most volatile unit economics because the number of LLM calls per task is unpredictable. A simple task might complete in 3 calls ($0.20), while a complex one might take 15 calls with retries ($3.00+). Without per-task cost tracking, average metrics mask dangerous outliers. CostHawk customers who tag requests with task_id typically discover that 5-10% of agent tasks account for 40-50% of total agent costs.

Chatbots have deceptively simple-looking economics that deteriorate over time. Early turns are cheap, but as conversation history accumulates, later turns send progressively more context tokens. A 12-turn conversation where each turn adds 300 tokens of history means turn 12 sends 3,600 tokens of history that turn 1 did not. Teams that quote per-query costs based on early-turn averages underestimate true session costs by 30-50%.

Code generation tools face a context window tax. Effective code generation requires sending relevant file contents, project structure, and coding conventions as context. A single completion request with 20,000 tokens of code context costs 10x more than a simple text generation with a 2,000-token prompt, even though both produce similar-length outputs. Tools that pre-fill large context windows pay a premium for quality.

Improving AI unit economics means either reducing the cost to serve each unit or increasing the revenue per unit. The highest-impact strategies, ordered by typical ROI and ease of implementation:

1. Implement Model Routing (Cost Reduction: 40-70%)

Not every query requires your most expensive model. A model router evaluates each incoming request and directs it to the cheapest model capable of producing an acceptable response. In practice, 50-70% of queries in most AI products can be handled by a smaller, cheaper model without noticeable quality degradation.

Example impact: A product routing 100% of queries to Claude 3.5 Sonnet at $3/$15 per million tokens switches to routing 60% to Claude 3.5 Haiku at $0.80/$4.00 per million tokens. Assuming identical query profiles, the blended inference cost drops by approximately 45%. CostHawk's per-model cost breakdowns make it easy to identify which query types are candidates for routing to cheaper models — look for high-volume, low-complexity queries that are currently being served by frontier models.

2. Optimize Prompts and Context (Cost Reduction: 20-50%)

System prompt bloat is the silent killer of unit economics. Most system prompts accumulate instructions over time as developers add edge case handling, persona guidance, and formatting rules. A structured audit typically finds that 30-50% of system prompt tokens can be removed without affecting output quality. At 100,000 queries per day, reducing a system prompt from 2,000 tokens to 1,000 tokens saves:

1,000 fewer tokens * 100,000 queries * $2.50/1M tokens = $0.25/day input savings
Annualized: ~$91/year just on input tokens for one prompt optimization

For products at scale (1M+ queries/day), the same optimization saves $912/year. Multiply across dozens of prompt templates and the savings become material.

Similarly, conversation history management dramatically affects per-session costs. Implementing a sliding window (keep last 5 turns instead of all turns) or summarizing older context can reduce later-turn costs by 50-70%.

3. Implement Prompt Caching (Cost Reduction: 15-40%)

Both Anthropic (90% discount on cached tokens) and OpenAI (50% discount) offer prompt caching for repeated prompt prefixes. If your system prompt is identical across requests — which it typically is — caching eliminates the majority of system prompt input costs. For a 2,000-token system prompt at 100,000 queries/day on Anthropic:

Without caching: 2,000 * 100,000 * $3.00/1M = $0.60/day
With caching (90% discount): $0.06/day
Savings: $0.54/day = $197/year

4. Right-Size Output Length (Cost Reduction: 15-30%)

Output tokens cost 4-5x more than input tokens. Setting appropriate max_tokens limits and instructing models to be concise reduces output costs without necessarily reducing quality. A product that generates 500-token responses when 250 tokens would suffice is paying double the necessary output cost. Add explicit length guidance to your prompts: "Respond in 2-3 sentences" or "Keep your response under 150 words."

5. Introduce Usage-Based Pricing (Revenue Increase: Variable)

If your unit economics are negative on a per-user basis because power users consume disproportionate resources, consider hybrid pricing: a base subscription fee plus usage-based charges above a threshold. This aligns revenue with cost and eliminates the "whale user" problem where a small number of heavy users destroy aggregate margins. Many AI products have moved to models like "$49/month includes 1,000 queries, $0.05 per additional query" to protect unit economics while maintaining accessible pricing for average users.

6. Batch and Deduplicate Requests (Cost Reduction: 10-25%)

Many AI products make redundant API calls: re-analyzing the same document when it has not changed, re-generating embeddings for content that was already embedded, or sending near-identical queries when a cached response would suffice. Implement semantic caching (cache responses for queries similar to previous ones) and content-hash deduplication to eliminate wasted inference spend. OpenAI's Batch API also offers a 50% discount for non-time-sensitive workloads.

Even well-designed AI products can experience unit economics breakdowns. Recognizing the warning signs early prevents small problems from becoming existential threats. Here are the six most common scenarios where AI unit economics deteriorate, along with detection and remediation strategies:

1. The Context Window Spiral

Symptom: Per-session costs increase 3-5x over a 3-month period without corresponding increases in session length or user count. Root cause: Developers gradually add more context to prompts — additional system instructions, larger document chunks in RAG, more conversation history — without measuring the cost impact. Each addition seems small, but they compound.

Detection: Monitor average input tokens per request over time in CostHawk. A steady upward trend without a corresponding product change is a red flag. Alert threshold: input tokens per request increasing more than 15% month-over-month.

Remediation: Audit all prompt templates and context injection logic. Set token budgets for each context component (system prompt: max 1,500 tokens; RAG context: max 3,000 tokens; conversation history: max 2,000 tokens). Enforce these budgets in code.

2. The Power User Subsidy

Symptom: Aggregate unit economics look healthy, but user-level analysis reveals that 8-12% of users account for 50-60% of total inference costs while paying the same subscription fee as everyone else. Root cause: flat-rate pricing with no usage caps or tiers.

Detection: In CostHawk, tag requests with user_id and generate a cost distribution report. If the Gini coefficient of per-user costs exceeds 0.6, you have a power user problem. Look for users whose monthly inference cost exceeds 2x their subscription price.

Remediation: Introduce usage tiers, rate limits, or usage-based pricing above a threshold. Alternatively, implement model routing that serves heavier users with cheaper models after they exceed a daily query budget.

3. The Agent Runaway

Symptom: Sudden, unpredictable cost spikes from agent workflows that make far more LLM calls than expected. A single user task might generate 20-50 API calls when the expected range is 3-8. Root cause: agentic loops without iteration limits, retry storms from transient API errors, or recursive tool-calling patterns.

Detection: Monitor the number of LLM calls per task/session. Set hard limits on maximum calls per agent execution (e.g., 15 calls max). Alert on any task that exceeds 2x the 95th percentile call count. CostHawk's anomaly detection can flag individual sessions whose cost exceeds 3x the rolling average.

Remediation: Implement circuit breakers that terminate agent loops after a configurable maximum number of iterations. Add cost circuit breakers that terminate execution if per-task cost exceeds a threshold (e.g., $2.00). Log and review terminated executions to improve agent reliability.

4. The Model Price Shock

Symptom: Monthly costs jump 20-40% without usage changes. Root cause: a provider changes pricing, or your codebase is updated to use a more expensive model variant (e.g., a library update that defaults to a newer, pricier model).

Detection: CostHawk tracks cost per token over time. A sudden change in effective price per token without a deliberate model change indicates either a pricing update or an unintended model switch. Pin model versions in your codebase and monitor for drift.

Remediation: Pin specific model versions in all API calls (e.g., gpt-4o-2024-11-20 instead of gpt-4o). Subscribe to provider pricing update announcements. Maintain a model pricing table (CostHawk does this automatically) and alert when effective rates diverge from expected rates.

5. The Feature That Kills Margins

Symptom: Unit economics were healthy at 45% margin, then a new feature launches and margins drop to 15% within two weeks. Root cause: the new feature requires expensive inference (large context, long output, frontier model) but was not priced or cost-modeled before launch.

Detection: Tag all requests with feature in CostHawk and monitor per-feature unit economics. Any feature with margins below 20% needs immediate attention. Require cost impact analysis as part of the feature development process — estimate per-query cost before building, then validate after launch.

Remediation: Cost-model every feature before development. For expensive features, consider gating them to higher pricing tiers, using cheaper models with acceptable quality tradeoffs, or implementing usage limits specific to that feature.

6. The Currency of Growth

Symptom: The product is growing rapidly — user count doubles in 3 months — but per-user revenue stays flat while per-user costs increase. New users are attracted by aggressive pricing or a generous free tier, and their usage patterns are heavier than early adopters. Root cause: growth-stage pricing that prioritizes acquisition over unit economics.

Detection: Segment unit economics by user cohort (signup month) in CostHawk. If newer cohorts have worse unit economics than older cohorts, growth is diluting margins. Also monitor the ratio of free-tier to paid-tier usage.

Remediation: Restructure the free tier to limit costly operations (cap queries, restrict model access, limit output length). Ensure paid tier pricing covers at least the 80th percentile of per-user cost. Some margin compression during hypergrowth is acceptable, but the trend line must show a path back to healthy margins as the product matures.