FAQ
Frequently Asked Questions
Common questions about LLM API pricing, tokens, caching, and cost optimization.
Data built Aug 10, 2026. Verify with providers before decisions.
What is a token in LLM API pricing?
A token is a unit of text that LLMs process. Roughly, 1 token is about 4 characters or 0.75 words in English. Input tokens are the text you send to the model, and output tokens are the text the model generates back. Pricing is typically shown per 1 million tokens.
Why do output tokens cost more than input tokens?
Output generation requires the model to run inference step-by-step for each token, while input processing can be parallelized. On average across 2,076 chat models, output tokens cost 3.6x more than input tokens.
What is prompt caching and how does it save money?
Prompt caching stores previously processed input tokens so they don't need to be re-processed. Providers like OpenAI, Anthropic, and Google offer up to 90% discount on cached input tokens. This is especially valuable for multi-turn conversations or RAG workloads where the same context is repeated. See the prompt caching guide for details.
What is batch API pricing?
Batch APIs let you submit jobs asynchronously and typically offer 50% off standard pricing. The trade-off is higher latency (results are returned within hours instead of seconds). Good for offline processing, data enrichment, or non-interactive workloads. See the cache and batch pricing guide for more.
How do I estimate my monthly LLM API cost?
Multiply your monthly input tokens by the input price per token, add output tokens times output price per token, and multiply by the number of requests. Use the calculator to estimate with real pricing data and workload presets, or try the compare tool to see models side by side.
When should I self-host instead of using an API?
Self-hosting becomes cost-effective when your monthly API spend exceeds $500-1000 and you have the engineering capacity to maintain infrastructure. Use the break-even calculator to compare your specific costs, or browse local models to see available open-weight options.
What is a context window and why does it matter?
The context window is the maximum number of tokens a model can process in a single request. Larger context windows allow more text, code, or data to be included, but input costs scale with context length. For example, a 128k-context request costs 4x more than a 32k-context request at the same per-token price. See the context window pricing guide for detailed comparisons.
What are reasoning models and why do they cost more?
Reasoning models (like OpenAI o3, Claude thinking, DeepSeek R1) use extended thinking to solve complex problems step-by-step. They cost 2-10x more than standard models because they generate many internal tokens before producing output. Use them for math, coding, and logic tasks where accuracy matters more than speed. See the reasoning pricing guide for details.
How do I compare models from different providers?
Use the compare tool to put 2-3 models side by side. You can compare pricing, context windows, modalities, and capabilities. Preset shortcuts help you find the cheapest chat, coding, or reasoning models across all providers. Browse the providers directory to see all available providers.
What is the model routing cascade?
Model routing cascade is a cost optimization strategy where you route simple queries to cheap models and complex queries to premium models. For example, a chatbot might route 80% of messages to a budget model and 20% to a premium model, cutting costs by 60-90% while maintaining quality for complex tasks. Read the model routing guide for a four-tier framework.
How much do embedding models cost?
Embedding models are much cheaper than chat models, typically $0.02-0.13 per million tokens. They convert text into numerical vectors for search and retrieval. Popular options include OpenAI text-embedding-3-small ($0.02/1M) and Cohere embed-v3 ($0.10/1M). See the embedding use case for cost examples.
What are open-weight models and local LLMs?
Open-weight models are models whose weights are publicly available, allowing you to run them on your own hardware. Popular families include Llama, Mistral, Qwen, and DeepSeek. Running them locally eliminates API costs but requires GPU hardware and engineering setup. See the local models page for available options and the local vs cloud comparison for cost analysis.
What is the cheapest LLM API for production use?
The cheapest LLM API depends on your workload. For simple chat tasks, GPT-4.1 Mini, Gemini 2.5 Flash, and Claude Haiku 4.5 offer the best value. For reasoning tasks, DeepSeek-R1 and Gemini 2.5 Flash offer lower costs than premium reasoning models. Use the calculator to estimate costs for your specific token volumes.
How do I choose between GPT-4o, Claude Sonnet, and Gemini Flash?
Each model has different strengths. GPT-4o offers strong all-around performance with vision and tool use. Claude Sonnet excels at coding and analysis with a large context window. Gemini Flash is optimized for speed and low cost. The best choice depends on your use case, budget, and latency requirements. See our provider guides for detailed comparisons.
How much does it cost to run an AI chatbot for 10,000 users?
Costs vary widely based on conversation length and model choice. For a simple chatbot with 500 tokens per conversation and 10 messages per user per day, expect $50-500/month depending on the model. Use the calculator with your specific token estimates for a more accurate projection.
What is the difference between OpenAI API and Azure OpenAI?
OpenAI API is the direct consumer API. Azure OpenAI is the enterprise version hosted on Microsoft Azure with additional security, compliance, and SLA guarantees. Pricing differs between the two, and Azure offers reserved capacity discounts. See our Azure vs OpenAI comparison for details.
How do LLM API costs scale with usage?
LLM API costs scale linearly with token usage. Most providers charge per input and output tokens. Volume discounts may apply at high usage levels. Caching and batch processing can reduce costs by 50-90%. Use the calculator to model cost scaling for your expected growth.
Have another question?
If you have a question not covered here, check the data sources page or use the calculator to estimate costs for your specific workload.