Webhook

Webhooks are a communication pattern where a server sends an HTTP POST request to a client-specified URL when an event occurs. The term was coined by Jeff Lindsay in 2007, combining "web" with "hook" (as in a programming hook — a point where custom code can be inserted into a process). Webhooks have become the standard mechanism for event-driven integrations across SaaS platforms, payment processors, version control systems, and monitoring tools.

The webhook lifecycle follows a simple pattern:

1. Registration: You provide the source system with a URL (your webhook endpoint) and specify which events you want to receive. For example, you might register https://api.yourcompany.com/webhooks/costhawk to receive cost.threshold.exceeded and anomaly.detected events.
2. Event occurrence: Something happens in the source system — a cost threshold is breached, a usage anomaly is detected, or a budget limit is approached.
3. Delivery: The source system constructs a JSON payload describing the event and sends an HTTP POST request to your registered URL. The request includes headers for authentication (typically an HMAC signature) and metadata.
4. Processing: Your endpoint receives the payload, verifies its authenticity, and executes whatever action is appropriate — posting to Slack, creating a Jira ticket, pausing an API key, or triggering a Lambda function.
5. Acknowledgment: Your endpoint returns an HTTP 200 status code to confirm receipt. If the source system receives a non-2xx response (or no response within a timeout), it retries the delivery according to its retry policy.

Webhooks are fundamentally different from APIs in their communication direction. With an API, your application pulls data by making requests on its own schedule. With a webhook, the source system pushes data to your application the moment it becomes relevant. This push model eliminates polling latency, reduces unnecessary API calls, and enables near-instantaneous response to events.

The key architectural advantage of webhooks is decoupling. The source system does not need to know what you do with the event data. It simply delivers the payload to your URL. You can change your processing logic — switching from Slack to Discord, adding a database write, or triggering an automated remediation — without any changes to the source system. This decoupling makes webhooks the most flexible and scalable integration pattern for event-driven workflows.

For AI cost management specifically, webhooks solve the fundamental timing problem: cost anomalies are time-sensitive events where every minute of delay increases financial exposure. Polling a dashboard every hour means you could miss up to 59 minutes of runaway spend. Webhooks deliver the alert within seconds of detection, compressing your response time from hours to minutes.

AI cost management generates several categories of events that benefit from real-time webhook delivery. Each event type serves a different operational purpose and typically routes to a different team or channel:

Cost Threshold Alerts: These fire when spending exceeds a defined dollar amount or percentage increase within a time window. Common configurations include:

• Hourly spend exceeds 2x the trailing 7-day hourly average
• Daily spend exceeds a fixed budget (e.g., $500/day)
• Weekly spend exceeds 80% of the monthly budget with more than 7 days remaining
• Per-key spend exceeds its allocated budget for the billing period

Cost threshold webhooks are the first line of defense against runaway spend. They should route to a high-visibility channel (Slack #cost-alerts or PagerDuty) and include the current spend amount, the threshold value, the percentage over threshold, and the affected resource (project, key, or model).

Anomaly Detection Alerts: These fire when CostHawk's anomaly detection algorithms identify statistically significant deviations from expected patterns. Unlike fixed thresholds, anomaly detection adapts to your baseline and catches subtle shifts that a static threshold might miss. Common anomaly types include:

• Sudden spike in request volume from a single API key
• Unusual increase in average tokens per request (suggesting prompt injection or context stuffing)
• Unexpected model usage (requests hitting an expensive model that should be routed to a cheaper one)
• Geographic or temporal anomalies (traffic from unexpected regions or at unusual hours)

Budget Warning Alerts: These provide advance warning before budgets are exhausted. Typical configurations include 50%, 75%, 90%, and 100% of budget consumption triggers. Budget warnings give teams time to adjust usage, request budget increases, or implement cost-saving measures before services are disrupted. CostHawk supports budgets at the organization, project, team, and individual API key level.

Usage Milestone Notifications: These fire when cumulative usage crosses defined thresholds — 1 million requests, 1 billion tokens, first usage of a new model, or first request from a new API key. Milestones are informational rather than urgent, typically routing to a general channel or weekly digest rather than paging an engineer.

System Health Events: These notify you of changes to the monitoring infrastructure itself — a wrapped key approaching its rate limit, a provider API returning elevated error rates, or a webhook endpoint failing to acknowledge deliveries. These meta-alerts ensure your monitoring pipeline stays healthy.

A mature AI cost monitoring setup typically configures 10–20 webhook rules across these categories, routing each to the appropriate channel and team based on urgency and ownership. CostHawk's webhook configuration interface lets you define rules declaratively, test them with sample payloads, and monitor delivery success rates from a single dashboard.

Webhooks become actionable when they reach the platforms where your team already works. Here is how CostHawk webhook payloads integrate with the most popular notification platforms:

Platform selection guidelines:

• For immediate human response (cost spikes, anomalies): Use PagerDuty or Opsgenie. These platforms provide on-call scheduling, escalation policies, and acknowledgment tracking that ensure critical alerts get seen and acted upon — even at 3 AM.
• For team awareness (budget warnings, daily summaries): Use Slack or Teams. These reach the entire team in a shared channel without the urgency of a page. Configure CostHawk to send rich, formatted messages with charts and drill-down links.
• For automation (auto-pause keys, auto-scale, auto-route): Use custom HTTP endpoints backed by serverless functions (AWS Lambda, Cloudflare Workers, or Vercel Edge Functions). These process the webhook payload programmatically and take automated action without human intervention.
• For compliance and audit: Route webhooks to a logging endpoint that stores every alert in a durable, append-only log. This provides an audit trail for cost governance and can be queried during incident reviews.

Most teams configure multiple destinations per event type — for example, a cost anomaly might simultaneously notify Slack (for team awareness), PagerDuty (for on-call response), and a Lambda function (for automated key throttling). CostHawk supports fan-out delivery to multiple endpoints per webhook rule.

A well-designed webhook payload contains everything the receiver needs to understand and act on the event without making additional API calls. CostHawk webhook payloads follow a consistent structure across all event types:

{
  "id": "evt_01HZ3K7M9N2P4Q5R6S7T8U9V0W",
  "type": "cost.threshold.exceeded",
  "created_at": "2026-03-16T14:32:07.000Z",
  "api_version": "2026-03-01",
  "data": {
    "alert_id": "alt_01HZ3K7M9N2P4Q5R6S7T8U9V0W",
    "alert_name": "Daily spend exceeds $500",
    "severity": "critical",
    "resource": {
      "type": "project",
      "id": "proj_abc123",
      "name": "Production Chatbot"
    },
    "threshold": {
      "metric": "daily_cost_usd",
      "operator": "greater_than",
      "value": 500.00,
      "window": "24h"
    },
    "current_value": 743.21,
    "percent_over": 48.6,
    "breakdown": {
      "by_model": [
        { "model": "gpt-4o", "cost": 512.40, "requests": 34200 },
        { "model": "claude-3.5-sonnet", "cost": 189.30, "requests": 8400 },
        { "model": "gpt-4o-mini", "cost": 41.51, "requests": 92100 }
      ],
      "by_key": [
        { "key_id": "key_xyz789", "key_name": "chatbot-prod", "cost": 623.80 },
        { "key_id": "key_def456", "key_name": "search-prod", "cost": 119.41 }
      ]
    },
    "trend": {
      "previous_day": 312.50,
      "seven_day_average": 298.73,
      "percent_change_vs_average": 148.8
    },
    "dashboard_url": "https://app.costhawk.com/dashboard/alerts/alt_01HZ3K7M9N2P4Q5R6S7T8U9V0W"
  }
}

Key design principles in this payload:

• Self-contained: The payload includes the alert name, threshold details, current value, breakdown, and trend data. A Slack integration can render a complete, actionable message without querying the CostHawk API for additional context.
• Typed and versioned: The type field enables receivers to route different event types to different handlers. The api_version field ensures backward compatibility as the payload schema evolves.
• Idempotent: The id field is a unique event identifier. Receivers should deduplicate on this field to handle retries gracefully — if the same event is delivered twice (due to a retry), processing it twice should not create duplicate alerts or actions.
• Actionable: The dashboard_url provides a direct link to investigate the alert in the CostHawk UI. The breakdown object identifies which models and keys are driving the cost spike, enabling targeted response.
• Severity-tagged: The severity field (critical, warning, info) lets receivers prioritize their response. A Slack integration might use red for critical, yellow for warning, and blue for info. A PagerDuty integration might only page for critical severity.

For anomaly detection events, the payload additionally includes the expected value, the standard deviation, and the z-score that triggered the anomaly, giving engineers the statistical context to assess whether the anomaly warrants action or is a benign fluctuation. For budget events, the payload includes the budget amount, consumed amount, remaining amount, projected exhaustion date, and burn rate.

Webhook delivery operates over HTTP, which means network failures, endpoint downtime, and processing errors can all prevent successful delivery. A robust webhook system must handle these failures gracefully to ensure no critical alert is lost.

Retry policies: When a webhook delivery fails (non-2xx response or timeout), CostHawk retries the delivery using an exponential backoff schedule:

After 6 failed retries spanning approximately 11 hours, the delivery is marked as failed and the event is logged in the webhook delivery history. CostHawk sends a separate notification (via email or a backup channel) when a webhook endpoint has been failing consistently, so you know your alert pipeline is degraded.

Timeout handling: CostHawk waits up to 10 seconds for your endpoint to respond. If your endpoint does not return an HTTP response within this window, the attempt is treated as a failure and the retry schedule begins. To avoid timeouts, your webhook endpoint should acknowledge receipt immediately (return 200) and process the payload asynchronously. A common pattern is to write the payload to a queue (SQS, Redis, or a database) and return 200 within milliseconds, then process the event in a background worker.

Idempotency: Because retries can deliver the same event multiple times, your webhook handler must be idempotent — processing the same event twice should produce the same result as processing it once. Use the id field in the webhook payload as a deduplication key. Before processing, check whether you have already handled an event with this ID. If so, return 200 without taking any action.

Signature verification: Every CostHawk webhook delivery includes an X-CostHawk-Signature header containing an HMAC-SHA256 signature of the payload, computed using your webhook signing secret. Always verify this signature before processing the payload to prevent malicious actors from sending fake webhook events to your endpoint:

import crypto from 'crypto'

function verifyWebhookSignature(
  payload: string,
  signature: string,
  secret: string
): boolean {
  const expected = crypto
    .createHmac('sha256', secret)
    .update(payload)
    .digest('hex')
  return crypto.timingSafeEqual(
    Buffer.from(signature),
    Buffer.from(expected)
  )
}

Monitoring webhook health: CostHawk tracks delivery success rate, average response time, and error codes for each webhook endpoint. If your endpoint's success rate drops below 95%, CostHawk flags it in the dashboard and sends a health alert. You can view delivery logs showing every attempt, the response code, response time, and payload size. This observability is critical for maintaining confidence that your alert pipeline is functioning correctly — the worst outcome is believing you are monitored when your webhook endpoint has been silently failing for days.

CostHawk provides a declarative webhook configuration system that lets you define, test, and monitor webhook endpoints from the dashboard or via the API. Here is how to set up a complete webhook-based alerting pipeline:

Step 1: Create a webhook endpoint. Navigate to Settings → Webhooks → Add Endpoint, or use the API:

POST /api/v1/webhooks
{
  "url": "https://api.yourcompany.com/webhooks/costhawk",
  "description": "Production cost alerts",
  "events": [
    "cost.threshold.exceeded",
    "anomaly.detected",
    "budget.warning",
    "budget.exhausted"
  ],
  "filters": {
    "projects": ["proj_abc123", "proj_def456"],
    "severity": ["critical", "warning"]
  }
}

CostHawk returns a webhook ID and a signing secret. Store the signing secret securely — you will use it to verify incoming payloads.

Step 2: Test the endpoint. Click "Send Test Event" in the dashboard or call the test endpoint. CostHawk sends a sample payload matching your configured event types so you can verify your endpoint receives, validates, and processes it correctly. The test payload includes a "test": true field so your handler can skip side effects during testing.

Step 3: Configure alert rules. Webhook endpoints receive events generated by alert rules. Configure rules in the Alerts section:

• Cost threshold rules: Define a metric (hourly, daily, or weekly spend), a comparison operator (greater than, percent increase over baseline), and a threshold value. Example: "Alert when daily spend for project Production Chatbot exceeds $500."
• Anomaly detection rules: Enable CostHawk's anomaly detection engine, which automatically establishes baselines and detects statistically significant deviations. Configure sensitivity (low, medium, high) to control the z-score threshold for triggering.
• Budget rules: Attach webhooks to budget milestones. Example: "Alert at 50%, 75%, 90%, and 100% of the $10,000 monthly budget for the Engineering team."

Step 4: Monitor delivery health. The Webhooks dashboard shows real-time delivery metrics for each endpoint:

• Success rate: Percentage of deliveries that received a 2xx response within the timeout window. Target: 99.5%+
• Average response time: How quickly your endpoint acknowledges deliveries. Target: under 500ms
• Recent failures: List of failed deliveries with response codes, timestamps, and retry status
• Event volume: Number of events delivered per hour/day, useful for understanding alert frequency and tuning thresholds to avoid alert fatigue

You can also use the CostHawk MCP server to manage webhooks from within your AI coding assistant. The costhawk_create_webhook and costhawk_list_webhooks tools let you configure and monitor webhooks without leaving your editor — a natural fit for engineering teams that live in their terminal.

Best practices for webhook configuration:

• Create separate endpoints for different severity levels — route critical alerts to PagerDuty and warnings to Slack
• Use filters to avoid noise — only subscribe to events for projects and severity levels you care about
• Set up a catch-all endpoint that logs all events to a database for audit and retrospective analysis
• Review and tune thresholds monthly — as your usage grows, static thresholds may need adjustment
• Test webhook endpoints after any infrastructure change (DNS, load balancer, firewall rules) to confirm continued connectivity