Prompt Caching Cost Savings Guide: Cut LLM API Bills by 75-90%

1. How Prompt Caching Works

When you send a request to an LLM API, the provider processes your input tokens. With prompt caching, the provider stores the initial prefix of your input (system prompt, few-shot examples, documents) on its servers. If the next request starts with the same prefix, the provider skips reprocessing those tokens and charges a lower cached-input rate.

The key constraint is exact prefix matching: the cached tokens must appear at the start of the prompt in the same order. Variations in system prompts, different document ordering, or even whitespace changes will miss the cache.

Pricing structure

Most providers use a three-tier pricing model for cached workloads:

• Standard input — the normal per-token rate for non-cached input
• Cached input — a discounted rate (typically 10-25% of standard) for tokens that hit the cache
• Cache write — an optional premium charged on the first request that populates the cache (some providers charge nothing extra)

Output tokens are always billed at the standard rate regardless of caching.

2. Provider Caching Comparison

Caching behavior and pricing vary by provider. Here is how the major providers compare:

The table below shows 982 models with cached input pricing across 39 providers in our catalog. Use the compare page to inspect cached pricing rows side by side.

3. Which Models Support Caching

The following table shows representative models from each major provider that support cached input pricing. For the full list, filter by "cached input" on the compare page.

4. Quantified Savings Examples

The real savings from caching depend on your cache hit rate — the percentage of requests where the input prefix matches the cached version. Here are concrete examples at different hit rates.

Scenario: 10k input tokens, 1k output tokens, 10,000 requests/month

This represents a typical RAG workload where the system prompt and retrieved context stay consistent across requests.

At an 80% cache hit rate, input costs drop by 72-90% depending on the provider. The savings are most dramatic for workloads with large system prompts or repeated document context.

5. Cost Scenarios: Cached vs Non-Cached Workloads

These three scenarios model real-world workloads where caching delivers measurable savings. Each uses the cache reuse preset from the calculator as a starting point.

6. Best Practices for Cache Optimization

Structure prompts for prefix stability

Place static content (system instructions, few-shot examples, reference documents) at the beginning of the prompt. Put variable content (user queries, conversation history) at the end. This maximizes the shared prefix across requests.

Minimize prefix variations

Avoid dynamic system prompts that change per request (timestamps, session IDs, random instructions). If you need dynamic elements, place them after the static prefix rather than before it.

Keep the cached prefix large enough

Each provider has a minimum prefix size for caching. OpenAI requires at least 1,024 tokens for GPT-4o models. Anthropic requires 2,048 tokens for Claude 3+. Google Gemini starts caching at just 32 tokens. Short prompts may not benefit from caching at all.

Monitor cache hit rates

OpenAI and Anthropic report cached tokens in their usage responses. Track this metric to understand your actual hit rate. If it is below 30%, the caching overhead may not be worth the implementation effort.

Account for cache TTL

Caches expire after a few minutes of inactivity. For low-traffic applications, the cache may expire between requests. Batch processing or scheduled jobs can benefit from caching if requests are clustered together.

Combine with model routing

Use a model routing cascade with caching: route simple queries to a budget model and complex queries to a premium model. Both routes benefit from cached system prompts, compounding the savings.

7. Frequently Asked Questions