Cost Anomaly Detection

Cost anomaly detection for AI APIs is a monitoring discipline that combines time-series analysis, statistical methods, and configurable alerting to identify when your AI spending deviates from normal patterns. It answers three questions continuously:

• Is my current spend normal? Comparing current hourly/daily cost against historical baselines to detect sudden increases.
• Is my spend trending in the right direction? Tracking week-over-week and month-over-month trends to catch gradual cost drift before it compounds.
• Are all my API keys/projects spending as expected? Comparing per-key and per-project costs against their individual baselines and against each other to detect localized anomalies.

The detection process typically works in four stages:

1. Data collection: Every API request is logged with its token counts, model, cost, API key, and metadata (tags, project, user). This creates the raw time-series data.
2. Baseline calculation: Historical data is aggregated into baselines — typically rolling 7-day and 30-day averages with standard deviations. Baselines account for known patterns like day-of-week variation (weekday traffic is often 2–3x weekend traffic) and time-of-day patterns.
3. Anomaly scoring: Current spending is compared against baselines using statistical methods (z-score, modified z-score, IQR). Each data point receives an anomaly score indicating how unusual it is.
4. Alert routing: Anomaly scores that exceed configured thresholds trigger alerts through the appropriate channel (Slack for warnings, PagerDuty for emergencies). Alerts include context: what changed, by how much, which keys/models are responsible, and when it started.

AI cost anomalies fall into three distinct categories, each requiring different detection approaches and response procedures:

1. Sudden Spikes

A sudden spike is a rapid, large increase in spending over minutes to hours. Common causes include:

• Runaway agent loops: An AI agent enters an infinite loop, making thousands of API calls. A single Claude 3.5 Sonnet agent loop running for 30 minutes can generate $500–$2,000 in charges.
• Traffic surges: A marketing campaign, press mention, or viral event drives 5–20x normal user traffic. If your app makes one API call per user action, costs scale proportionally.
• Model upgrade deployment: A developer deploys code that switches from GPT-4o-mini ($0.15/$0.60 per MTok) to GPT-4o ($2.50/$10.00 per MTok), increasing per-request cost by 16x. If this deploys during peak hours, the cost impact is immediate and severe.
• Retry storms: A downstream service outage triggers aggressive retries without exponential backoff, multiplying request volume 5–10x while also failing (and being billed for input tokens on failed requests that return errors).

Detection method: Compare current 15-minute and 1-hour spending against the same time window's baseline. Alert when spending exceeds 2x the baseline (warning) or 5x (critical).

2. Gradual Drift

Gradual drift is a slow, sustained increase in costs over days to weeks. It is harder to detect because no single data point looks anomalous. Common causes:

• Prompt bloat: System prompts grow as developers add instructions, examples, and edge case handling. A prompt that starts at 500 tokens might grow to 3,000 tokens over 6 weeks — a 6x increase in per-request input cost that happens so gradually no one notices.
• Conversation length creep: Users discover that longer conversations produce better results, so average conversation length increases from 3 turns to 8 turns. Each turn includes all previous turns as input, so the cost per conversation grows quadratically.
• Feature adoption: A new AI-powered feature launches with low adoption (5% of users). Over 4 weeks, adoption grows to 40%. Each percentage point of adoption adds incremental API cost that looks normal day-to-day but compounds to a 8x increase.

Detection method: Compare 7-day rolling average against 30-day rolling average. Alert when the 7-day average exceeds the 30-day average by more than 25% (warning) or 50% (critical).

3. Per-Key Anomalies

Per-key anomalies occur when a specific API key, project, or team deviates from its own baseline or from its peers — even if total spending looks normal. Common causes:

• Compromised key: A leaked API key is being used by unauthorized parties. Their usage adds to your bill and may not follow normal usage patterns (e.g., requests at 3 AM from unusual geographies).
• Testing in production: A developer runs a load test or data migration using a production API key, generating thousands of extra requests.
• Single-tenant spike: One customer of your SaaS product starts using an AI feature 50x more than average, causing their allocated key to spike while others remain stable.

Detection method: Compare each key's current spending against its own historical baseline AND against the median of all similar keys. Alert when a single key's spending exceeds 3x its own baseline or 5x the peer median.

Several statistical and algorithmic approaches power cost anomaly detection, ranging from simple threshold rules to machine learning models. Here are the most common, in order of complexity:

1. Static Thresholds

The simplest approach: alert when spending exceeds a fixed dollar amount. Example: "Alert if daily spend exceeds $500." Easy to implement but brittle — does not adapt to growth, misses anomalies below the threshold, and fires false positives after legitimate traffic increases.

2. Moving Averages

Compare current spending against a rolling average (typically 7 or 30 days). Alert when current spend deviates from the average by more than a configurable percentage. This adapts to organic growth but is slow to respond to sudden shifts and can be skewed by outliers in the baseline window.

// Simple moving average anomaly detection
const WINDOW_DAYS = 7;
const THRESHOLD = 2.0; // Alert at 2x baseline

function checkAnomaly(currentDailySpend: number, last7Days: number[]): boolean {
  const avg = last7Days.reduce((a, b) => a + b, 0) / last7Days.length;
  return currentDailySpend > avg * THRESHOLD;
}

3. Z-Score (Standard Deviation)

Calculate how many standard deviations the current value is from the mean. A z-score above 2.0 means the value is in the top 2.3% of expected values — likely anomalous. Above 3.0, it is in the top 0.1% — almost certainly anomalous. This is CostHawk's default detection method because it balances sensitivity with false-positive rates.

// Z-score anomaly detection
function zScoreAnomaly(current: number, history: number[]): { score: number; isAnomaly: boolean } {
  const mean = history.reduce((a, b) => a + b, 0) / history.length;
  const stdDev = Math.sqrt(
    history.reduce((sum, val) => sum + Math.pow(val - mean, 2), 0) / history.length
  );
  const zScore = stdDev === 0 ? 0 : (current - mean) / stdDev;
  return { score: zScore, isAnomaly: zScore > 2.5 };
}

4. Modified Z-Score (MAD-based)

Uses median absolute deviation (MAD) instead of standard deviation, making it robust to outliers in the baseline. If your spending history includes a few spike days, standard z-score's baseline gets inflated, making future spikes harder to detect. MAD-based scoring is not affected by these outliers.

5. Seasonal Decomposition

Decomposes the spending time series into trend, seasonal (day-of-week, time-of-day), and residual components. Anomaly detection runs on the residual component only, avoiding false positives from predictable patterns. Essential for teams with strong weekday/weekend traffic differences.

6. Machine Learning (Isolation Forest, LSTM)

ML approaches learn complex patterns in spending data and can detect subtle anomalies that statistical methods miss. Isolation Forest is popular for its simplicity and effectiveness. LSTM neural networks can model temporal dependencies for very sophisticated detection. The trade-off is complexity — ML models need training data, hyperparameter tuning, and ongoing monitoring of their own performance. Recommended only for teams spending $50,000+/month where the ROI justifies the engineering investment.

Not all anomalies require the same response. A well-designed alerting system uses tiered severity levels with escalating actions:

Key design principles for alerting:

• Alert fatigue is real: If your team receives more than 5 non-actionable alerts per week, they will start ignoring all alerts. Tune thresholds to minimize false positives — start conservative (high thresholds) and lower them gradually.
• Context in every alert: An alert that says "spending is high" is useless. Every alert should include: current spend vs baseline, percentage deviation, the top 3 contributing factors (key, model, endpoint), when the anomaly started, and a link to the detailed investigation dashboard.
• Escalation paths: Info and warning alerts go to the team Slack channel. Critical alerts page the on-call engineer. Emergency alerts page the engineering manager and auto-remediate. This ensures the right person responds at the right urgency.
• Auto-remediation: For emergency-tier anomalies, waiting for human response is too slow. Configure automatic actions: disable a specific key, throttle traffic to a specific endpoint, or switch to a cheaper model. CostHawk's webhook integration enables these automated responses.

Here is a practical implementation of a cost anomaly detection system using z-score analysis on hourly spending data. This can run as a cron job, a serverless function, or as part of your existing monitoring pipeline:

interface SpendingRecord {
  timestamp: Date;
  totalCost: number;
  model: string;
  apiKey: string;
}

interface AnomalyAlert {
  severity: 'info' | 'warning' | 'critical' | 'emergency';
  currentSpend: number;
  baselineAvg: number;
  zScore: number;
  topContributors: { key: string; cost: number; pctOfTotal: number }[];
  message: string;
}

function detectAnomalies(
  currentHourSpend: number,
  historicalHourly: number[],  // Same hour, last 30 days
  perKeySpend: Map<string, number>,
  perKeyBaselines: Map<string, { mean: number; stdDev: number }>
): AnomalyAlert | null {
  // Calculate z-score for total spend
  const mean = historicalHourly.reduce((a, b) => a + b, 0) / historicalHourly.length;
  const stdDev = Math.sqrt(
    historicalHourly.reduce((sum, v) => sum + Math.pow(v - mean, 2), 0) / historicalHourly.length
  );
  const zScore = stdDev === 0 ? 0 : (currentHourSpend - mean) / stdDev;

  // Determine severity
  let severity: AnomalyAlert['severity'] | null = null;
  if (zScore > 4.0) severity = 'emergency';
  else if (zScore > 3.0) severity = 'critical';
  else if (zScore > 2.0) severity = 'warning';
  else if (zScore > 1.5) severity = 'info';
  else return null;  // Normal spending

  // Find top contributors
  const contributors = Array.from(perKeySpend.entries())
    .map(([key, cost]) => ({ key, cost, pctOfTotal: cost / currentHourSpend * 100 }))
    .sort((a, b) => b.cost - a.cost)
    .slice(0, 3);

  return {
    severity,
    currentSpend: currentHourSpend,
    baselineAvg: mean,
    zScore: Math.round(zScore * 100) / 100,
    topContributors: contributors,
    message: `Spending $${currentHourSpend.toFixed(2)}/hr vs baseline $${mean.toFixed(2)}/hr (z=${zScore.toFixed(1)})`,
  };
}

This implementation detects anomalies at the aggregate level and identifies the top-spending keys for investigation. In production, you would also run per-key z-score analysis to catch localized anomalies that might not show up in total spend (e.g., one key tripling while others drop, keeping total spend flat).

To avoid false positives during expected traffic changes (product launches, Black Friday, marketing campaigns), implement a "scheduled exception" system that temporarily raises baselines for specific date ranges. CostHawk supports this via its maintenance window feature — mark a time range as "expected increase" and the anomaly detector automatically adjusts thresholds.

CostHawk provides enterprise-grade cost anomaly detection out of the box, requiring no statistical expertise or custom implementation. Here is what CostHawk's anomaly detection system includes:

Real-time detection: Every API request processed through CostHawk's proxy or SDK is analyzed in real time. Anomalies are detected within 5 minutes of onset — compared to the industry average of 11 days for teams relying on provider billing dashboards.

Multi-dimensional analysis: CostHawk does not just monitor total spend. It runs anomaly detection across six dimensions simultaneously: total spend, per-model spend, per-key spend, per-project spend, per-tag spend, and request volume. This catches anomalies that are invisible in any single dimension — for example, a model switch that keeps total request volume constant but increases per-request cost 5x.

Adaptive baselines: Baselines automatically adjust for day-of-week patterns, time-of-day patterns, and organic growth trends. You do not need to manually recalibrate thresholds as your usage grows. CostHawk uses seasonal decomposition to separate predictable variation from true anomalies.

Smart alert routing: Configure alert destinations per severity tier. Info alerts go to your cost-tracking Slack channel. Warning alerts add the team lead. Critical alerts fire a PagerDuty incident. Emergency alerts trigger auto-remediation (key disable, traffic throttling) via webhooks to your API gateway.

Root cause analysis: When an anomaly fires, CostHawk's alert includes an automated root cause analysis: which keys, models, and endpoints are driving the spike, when it started, the estimated financial impact if it continues, and recommended actions. This reduces mean-time-to-resolution from hours to minutes.

Anomaly history: All detected anomalies are logged with their resolution status and root cause. This creates an institutional knowledge base that helps teams understand their cost risk profile and improve their prevention measures over time. Monthly anomaly reports show trends in frequency, severity, and root causes.