API Key Management

API key management is the discipline of controlling the full lifecycle of API credentials — from creation through revocation — to ensure that keys are secure, properly scoped, traceable, and compliant with organizational policies. For AI API keys specifically, this discipline includes cost-aware practices that traditional API key management does not address.

The lifecycle of an AI API key has six stages:

1. Creation. Keys are generated at the provider (OpenAI, Anthropic, Google) with specific scopes and labels. Best practice is to create separate keys for each team, service, and environment (development, staging, production) to enable granular cost attribution and limit blast radius.
2. Storage. Keys must be stored encrypted at rest. Never store API keys in plaintext in configuration files, environment variables on shared systems, or source code. Use a secrets manager (AWS Secrets Manager, HashiCorp Vault, 1Password) or an encrypted environment variable system. CostHawk encrypts all stored provider keys with AES-256-CBC using unique initialization vectors per key.
3. Distribution. Keys must reach the applications that need them without being exposed in transit. Inject keys through CI/CD pipeline secrets, Kubernetes secrets, or runtime secrets managers — never through chat messages, email, or shared documents. Each team member should have access only to the keys they need (principle of least privilege).
4. Monitoring. Track key usage in real time: which key is making requests, to which models, at what volume, and at what cost. CostHawk's wrapped-key system provides this monitoring automatically — every request is logged with the wrapped key ID, model, token counts, and computed cost.
5. Rotation. Replace keys periodically (quarterly at minimum, monthly for high-value keys) to limit the window of exposure if a key is compromised. Rotation must be seamless — the new key must be deployed to all consuming services before the old key is revoked.
6. Revocation. Immediately disable keys that are compromised, no longer needed, or associated with departed employees. Provider dashboards allow instant revocation, but you must also update all services that used the revoked key to prevent service disruptions.

Most organizations handle stages 1 and 2 reasonably well (because developers need keys to work and know not to commit them to Git), but stages 3–6 are often neglected. The result is a sprawl of untracked, unrotated, over-permissioned keys that create both security and cost liabilities.

API keys are the natural boundary for cost attribution in AI infrastructure. Every API request is authenticated with a key, and the provider's billing system tracks usage per key (or per organization/project, depending on the provider). By controlling how keys are created and assigned, you control the granularity of your cost data.

Per-key cost tracking enables:

• Team-level chargebacks. Issue separate keys (or CostHawk wrapped keys) to each team. Monthly cost reports show exactly how much each team spent, enabling budget accountability. Engineering Team A spent $12,000 on Claude 3.5 Sonnet for code review; Product Team B spent $8,000 on GPT-4o for customer chatbots. Without per-key tracking, these costs are a single $20,000 line item with no attribution.
• Service-level optimization. When each microservice has its own key, you can identify which services are the biggest cost drivers and target them for optimization. A common discovery is that a single service — often a batch processing pipeline or internal tool — accounts for 40–60% of total AI spend despite serving a small fraction of users.
• Environment cost separation. Separate keys for development, staging, and production environments reveal how much of your AI spend is non-production. CostHawk customers typically find that development and testing account for 20–35% of total AI API costs — an immediate optimization target. Reducing dev environment spending through model downgrading, request capping, or mock responses can save thousands per month.
• Anomaly detection. Per-key usage baselines enable cost anomaly detection. If a production key that normally processes 50,000 requests per day suddenly jumps to 500,000, CostHawk flags the anomaly and alerts the key owner. Without per-key tracking, this spike is buried in aggregate metrics and may not be noticed until the monthly bill arrives.

Abuse prevention. Unmanaged keys are a cost liability because they can be used without accountability. A developer running personal experiments on a shared production key, a contractor retaining key access after their engagement ends, or a compromised key being used by an external attacker — all of these scenarios generate real costs with no legitimate business value. Per-key management with budget caps, usage alerts, and regular access reviews prevents these scenarios. CostHawk wrapped keys can be configured with daily and monthly spending limits that hard-stop requests once the budget is exhausted, preventing any single key from generating unbounded costs.

These eight practices represent the standard of care for AI API key management in production environments. Implementing all eight eliminates the most common sources of key-related security incidents and cost overruns.

1. One key per scope. Never share a single API key across multiple teams, services, or environments. Create separate keys for each logical boundary. At minimum, maintain separate keys for: each production service, each non-production environment, each team or business unit, and each external integration. This limits blast radius (a compromised key only affects one scope) and enables granular cost attribution.

2. Least privilege. Scope keys to the minimum permissions required. If a service only needs access to GPT-4o mini, do not give it a key that can access GPT-4o or o1. Most providers do not offer model-level key scoping natively, but proxy layers like CostHawk wrapped keys can enforce model restrictions at the proxy level — requests for unauthorized models are rejected before reaching the provider.

3. Encrypt at rest. Store all API keys using a secrets manager with encryption at rest. AWS Secrets Manager, Google Cloud Secret Manager, HashiCorp Vault, and 1Password all provide this capability. Never store keys in plaintext environment files on shared servers, CI/CD configuration visible to all developers, or application configuration files — even if those files are in private repositories.

4. Rotate regularly. Rotate keys every 90 days at minimum, every 30 days for high-value production keys. Rotation should be automated: a scheduled job generates a new key at the provider, updates the secrets manager, deploys the new key to consuming services, and revokes the old key after a grace period (24–48 hours to ensure all services have picked up the new key).

5. Monitor usage. Track every request made with every key. Alert on anomalies: sudden usage spikes, usage from unexpected IP addresses, usage of models that the key's owner should not be using, and usage outside of normal business hours. CostHawk provides this monitoring out of the box for wrapped keys, with configurable alerts for cost thresholds, usage spikes, and unusual patterns.

6. Audit access. Maintain a log of who has access to which keys, when access was granted, and by whom. Review access quarterly: revoke keys for departed employees, decommissioned services, and completed projects. The access review should be a formal process, not a best-effort cleanup.

7. Respond to compromises immediately. If a key is suspected of being compromised — exposed in a public repository, included in a log file, or sent via insecure channel — revoke it immediately and generate a replacement. Do not wait to confirm the compromise; the cost of a false positive (brief service disruption during key swap) is far less than the cost of a real compromise (unauthorized charges, data exposure).

8. Budget caps per key. Set spending limits on every key. A production key might have a $5,000/day cap; a development key might have a $50/day cap. These caps prevent runaway costs from bugs, abuse, or compromises. CostHawk wrapped keys support configurable budget caps with hard enforcement — the proxy rejects requests once the cap is reached.

Per-key cost attribution is the practice of tracking AI API costs at the individual API key level, enabling precise allocation of costs to teams, projects, features, and environments. This is the foundation of AI cost governance — without it, the monthly AI bill is an opaque aggregate that cannot be optimized because no one knows which component is responsible for which portion.

How per-key attribution works:

At the provider level, most providers track usage per API key or per organization/project:

• OpenAI tracks usage per organization and per project. You can create multiple projects within an organization, each with its own API key and usage dashboard.
• Anthropic tracks usage per workspace. Separate workspaces have separate billing and usage tracking.
• Google tracks usage per GCP project with detailed billing exports to BigQuery for analysis.

However, provider-level tracking has limitations: it cannot cross-reference usage across providers, it cannot attribute costs to specific features or code paths within a service, and the granularity depends on how many keys/projects/workspaces you create at the provider level.

CostHawk's wrapped-key system adds a layer of attribution on top of provider-level tracking:

// Team A's wrapped key for their code review service
const teamAClient = new OpenAI({
  apiKey: "ch_team_a_code_review_sk_...",
  baseURL: "https://proxy.costhawk.com/v1"
})

// Team B's wrapped key for their customer chatbot
const teamBClient = new OpenAI({
  apiKey: "ch_team_b_chatbot_sk_...",
  baseURL: "https://proxy.costhawk.com/v1"
})

// Both use the same underlying provider key,
// but CostHawk tracks costs separately per wrapped key

This approach has several advantages over provider-level attribution:

• Unlimited granularity. Create as many wrapped keys as you need — per team, per service, per feature, per environment — without creating separate provider accounts or projects.
• Cross-provider aggregation. CostHawk normalizes costs across all providers, so you can see that Team A spent $8,000 total ($5,000 on OpenAI + $3,000 on Anthropic) without manually aggregating across provider dashboards.
• Real-time tracking. Costs are computed per-request and available in the dashboard within seconds, rather than waiting for end-of-month billing statements.
• Tagging and metadata. Wrapped keys can be tagged with arbitrary metadata (team name, project code, cost center) that flows into analytics and reporting. This enables finance-oriented views like cost-per-cost-center or cost-per-customer.

The operational impact of per-key attribution is significant. When teams can see their own spending and compare it to their budget, they self-optimize. CostHawk customers report a 15–20% reduction in AI spend within the first quarter of enabling per-key dashboards, driven entirely by teams identifying and eliminating their own waste — no top-down mandates required.

Key rotation — periodically replacing API keys with new ones — is a critical security practice that limits the window of exposure if a key is compromised. The challenge is rotating keys without disrupting the services that depend on them. Here is a zero-downtime rotation procedure for AI API keys:

Step 1: Generate the new key. Create a new API key at the provider with the same permissions and scopes as the existing key. Do not revoke the old key yet — both keys must be valid simultaneously during the transition.

Step 2: Update the secrets manager. Store the new key in your secrets manager alongside the old key. Tag the new key as "pending rotation" and record the rotation timestamp for audit purposes.

Step 3: Deploy the new key to all consuming services. Update environment variables, Kubernetes secrets, or configuration files in all services that use this key. For containerized deployments, this typically means updating the secret and triggering a rolling restart. For serverless functions, update the environment variable in the deployment configuration.

# Kubernetes secret update (zero-downtime with rolling restart)
kubectl create secret generic openai-key \
  --from-literal=OPENAI_API_KEY="sk-new-key-abc123" \
  --dry-run=client -o yaml | kubectl apply -f -

# Trigger rolling restart of deployments that use this secret
kubectl rollout restart deployment/chatbot-service
kubectl rollout restart deployment/code-review-service

Step 4: Verify the new key is in use. Monitor API request logs to confirm that all services are now authenticating with the new key. CostHawk's per-key usage dashboard makes this easy — you should see traffic shifting from the old wrapped key to the new one over the course of the rolling restart.

Step 5: Grace period. Wait 24–48 hours after all services have migrated to the new key. This grace period catches edge cases: long-running batch jobs that cached the old key in memory, services in other time zones that deploy on a different schedule, and any services you may have missed in Step 3.

Step 6: Revoke the old key. After the grace period, revoke the old key at the provider and remove it from the secrets manager. Record the revocation in your audit log.

Automating rotation with CostHawk wrapped keys. CostHawk wrapped keys simplify rotation because the wrapped key itself does not change — only the underlying provider key is rotated. Your services continue to use the same wrapped key (ch_team_a_sk_...) while CostHawk's backend swaps the underlying provider key. This means zero configuration changes in consuming services: update the provider key in CostHawk's dashboard, and all traffic through that wrapped key immediately uses the new provider key. This is the lowest-friction rotation model available and is one of the primary operational benefits of the proxy architecture.

CostHawk provides a comprehensive key management layer on top of your existing provider API keys, combining security best practices with cost tracking and policy enforcement. Here is how CostHawk's key management capabilities map to the key lifecycle:

Key onboarding. When you add a provider API key to CostHawk, it is encrypted with AES-256-CBC using a unique initialization vector and stored in CostHawk's secure database. The raw key is never logged, never included in API responses, and never accessible to CostHawk support staff. You can add keys from multiple providers (OpenAI, Anthropic, Google) and manage them all from a single dashboard.

Wrapped key generation. For each provider key, you can generate multiple wrapped keys with different scopes and policies:

• Team-scoped keys: ch_engineering_sk_..., ch_product_sk_..., ch_data_science_sk_... — each team gets its own cost tracking namespace.
• Service-scoped keys: ch_chatbot_prod_sk_..., ch_code_review_prod_sk_... — each service has independent budget and monitoring.
• Environment-scoped keys: ch_dev_sk_..., ch_staging_sk_..., ch_prod_sk_... — separate environments with different budget caps (low caps for dev, higher for production).

Policy enforcement. Each wrapped key can be configured with policies that the proxy enforces on every request:

Usage monitoring and alerts. CostHawk tracks every request per wrapped key in real time. The dashboard shows: total spend per key (daily, weekly, monthly), token consumption by model, request volume and error rates, latency percentiles, and cost anomaly detection. You can configure alerts for: daily spend exceeding a threshold, sudden usage spikes (2x or more above the 7-day average), new models being accessed by a key for the first time, and keys with zero usage for 30+ days (candidates for revocation).

Rotation support. When you rotate the underlying provider key, update it in CostHawk's dashboard and all wrapped keys immediately use the new provider key. No changes needed in any consuming service. CostHawk also tracks provider key age and can alert you when a key is overdue for rotation based on your configured rotation policy (30, 60, or 90 days).

Audit trail. Every administrative action is logged with the actor, timestamp, and details: key creation, policy changes, budget modifications, key revocation. This audit trail supports compliance requirements (SOC 2, HIPAA, SOX) and provides forensic data for incident investigation.