LLM API cost cascade: how model routing can cut your AI bill by up to 90%

The one-number case for cascading

Take a simple classification task — a support ticket router, content moderator, or intent classifier — consuming roughly 2,000 input tokens and 500 output tokens per request.

• Budget model (e.g. Qwen2.5-Coder-3B-Instruct): $35.00 per million tasks
• Premium model (e.g. gemini-2.5-pro): $7500.00 per million tasks

The premium model costs 214x more for this task. If your application handles one million simple classification requests per month, switching them to a budget model saves $7465/month with no quality loss for the task.

How much do budget models actually cost?

Of 2092 priced chat models in the database, 745 cost less than $1 per million output tokens. Many of these models handle classification, extraction, summarization, and simple Q&A with excellent results.

Per-million-task costs let you scale linearly: double the volume, double the cost.

How much do premium models cost for the same work?

248 models cost $10 or more per million output tokens. These are the frontier models that excel at complex reasoning, code generation, and nuanced analysis — but they are overkill for the majority of day-to-day API calls.

The cascade strategy

A cascade (or routing) strategy classifies each request by complexity and routes it to the cheapest model that can handle it reliably. The model routing cascade guide defines four tiers:

1. Budget (< $1/M output): Simple classification, extraction, formatting — ~745 models
2. Mid ($1–$10/M output): General Q&A, content generation, structured output
3. Premium ($10–$30/M output): Complex reasoning, code, nuanced analysis — ~248 models
4. Reasoning (thinking tokens): Multi-step planning, math, research — ~657 models

If 80% of your requests can be handled by budget models, 15% by mid-range, and 5% by premium or reasoning, your cost structure shifts dramatically.

In this example, cascading saves ~95% compared to using premium models for every request.

Beyond routing: complementary savings

Model cascading is the biggest lever, but three complementary strategies compound the savings:

• Prompt caching (−50–90% on input): The cache pricing guide shows how cached reads drop to $0.10/M on Anthropic and $0.0036/M on DeepSeek. For multi-turn conversations and RAG workloads, caching can halve your input costs.
• Batch processing (−50%): OpenAI and Anthropic offer 50% discounts for requests processed within a 24-hour window. Offline data labeling, nightly content generation, and bulk analysis are good candidates.
• Output token control: Output tokens cost 3–5x more than input across all providers. Trimming system prompts, setting max_tokens, and using structured output formats reduces the expensive half of each request.

Together, these four strategies can reduce a typical production LLM bill by 70–90%.

When not to cascade

Cascading is not free. Each routing decision adds latency (typically 50–200ms for a classification step), and a misrouted request may need a retry. Scenarios where cascading adds little value:

• Low-volume workloads (< 10k requests/month): The savings are small enough that one pipeline is simpler.
• Single-purpose agents: A fine-tuned budget model may handle the whole task without routing.
• Quality-sensitive apps: Medical diagnosis, legal analysis — use the most capable model and skip routing complexity.
• Real-time latency requirements: A classification pre-step adds overhead; for sub-500ms responses, consider a single fast model.

Next steps

See the actual numbers for your workload:

• Open the cost calculator with your token counts, volume, and cache assumptions
• Compare a budget vs premium preset with live pricing data
• Read the full cascade framework for each tier’s cutoff logic