LLM API cost cascade: how model routing can cut your AI bill by up to 90%

The one-number case for cascading

Take a simple classification task — a support ticket router, content moderator, or intent classifier — consuming roughly 2,000 input tokens and 500 output tokens per request.

• Budget model (e.g. Qwen2.5-Coder-7B): $35.00 per million tasks
• Premium model (e.g. gemini-2.5-pro): $7500.00 per million tasks

The premium model costs 214x more for this task. If your application handles one million simple classification requests per month, the hypothetical price difference between these routes is $7465/month as a price-only estimate before caching, batching, tools, or other charges. Evaluate classification accuracy, false positives and negatives, and escalation rates on held-out examples before assuming it preserves acceptable quality.

Try routing cascade in calculator →

How much do budget models actually cost?

Of 2107 priced chat models in the database, 741 have output prices at or below $1 per million tokens. Pricing and chat-mode metadata do not establish suitability or accuracy for classification, extraction, or summarization.

Per-million-task costs let you scale linearly: double the volume, double the cost.

How much do premium models cost for the same work?

229 models cost $10 or more per million output tokens. These are the frontier models that excel at complex reasoning, code generation, and nuanced analysis — but they are overkill for the majority of day-to-day API calls.

The cascade strategy

A cascade (or routing) strategy classifies each request by complexity and routes it to the cheapest model that can handle it reliably. The model routing cascade guide defines four tiers:

1. Budget (< $1/M output): Simple classification, extraction, formatting — ~741 models
2. Mid ($1–$10/M output): General Q&A, content generation, structured output
3. Premium ($10–$30/M output): Complex reasoning, code, nuanced analysis — ~229 models
4. Reasoning (thinking tokens): Multi-step planning, math, research — ~704 models

If 80% of your requests can be handled by budget models, 15% by mid-range, and 5% by premium or reasoning, your cost structure shifts dramatically.

In this example, cascading saves ~95% compared to using premium models for every request.

Beyond routing: complementary savings

Model cascading is the biggest lever, but three complementary strategies compound the savings:

• Prompt caching (−50–90% on input): The cache pricing guide shows how cached reads drop to $0.10/M on Anthropic and $0.0036/M on DeepSeek. For multi-turn conversations and RAG workloads, caching can halve your input costs.
• Batch processing (−50%): OpenAI and Anthropic offer 50% discounts for requests processed within a 24-hour window. Offline data labeling, nightly content generation, and bulk analysis are good candidates.
• Output token control: Output tokens cost 3–5x more than input across all providers. Trimming system prompts, setting max_tokens, and using structured output formats reduces the expensive half of each request.

Together, these four strategies can reduce a typical production LLM bill by 70–90%.

When not to cascade

Cascading is not free. Each routing decision adds latency (typically 50–200ms for a classification step), and a misrouted request may need a retry. Scenarios where cascading adds little value:

• Low-volume workloads (< 10k requests/month): The savings are small enough that one pipeline is simpler.
• Single-purpose agents: A fine-tuned budget model may handle the whole task without routing.
• Quality-sensitive apps: Medical diagnosis, legal analysis — use the most capable model and skip routing complexity.
• Real-time latency requirements: A classification pre-step adds overhead; for sub-500ms responses, consider a single fast model.

Next steps

See the actual numbers for your workload:

• Open the cost calculator with your token counts, volume, and cache assumptions
• Compare a budget vs premium preset with live pricing data
• Read the full cascade framework for each tier’s cutoff logic