Reference
LLM API Pricing Glossary
Key terms and definitions for understanding LLM API pricing, model comparison, and cost optimization. Use this glossary when reading articles, comparing models, or estimating costs.
Pricing & Billing
Per 1M tokens
The standard pricing unit for LLM APIs. Prices are expressed as USD per one million tokens processed. Input and output tokens are typically priced separately.
Related: Input tokens, Output tokens, Pricing
Input tokens
Tokens sent to the model in your prompt. Includes system instructions, conversation history, context documents, and user queries. Priced per 1M tokens.
Related: Output tokens, Context window, Prompt caching
Output tokens
Tokens generated by the model in response. Typically priced 2-5x higher than input tokens because generation requires more compute per token.
Related: Input tokens, Output-to-input ratio, Reasoning models
Batch pricing
Discounted pricing for async, non-urgent requests submitted in batch. Typically 50% cheaper than real-time pricing. Results are returned within hours, not seconds.
Related: Priority pricing, Flex pricing, Batch processing models
Priority pricing
Premium pricing for requests that need faster processing than standard. Often includes higher rate limits and guaranteed response times.
Related: Batch pricing, Flex pricing, Rate limits
Flex pricing
Variable pricing that adjusts based on demand. Lower costs during off-peak hours, higher during peak usage. Good for non-urgent workloads.
Related: Batch pricing, Priority pricing
Rate limits
Restrictions on how many requests you can make per minute (RPM) or how many tokens per minute (TPM). Limits increase with higher spending tiers.
Related: RPM, TPM, Pricing tiers
RPM
Requests Per Minute. The maximum number of API calls you can make in one minute. Varies by model, provider, and spending tier.
Related: TPM, Rate limits, Pricing tiers
TPM
Tokens Per Minute. The maximum number of tokens you can process in one minute across all requests. Higher TPM allows faster throughput.
Related: RPM, Rate limits, Pricing tiers
Context & Caching
Context window
The maximum number of tokens a model can process in a single request, including both input and output. Larger context windows allow more data but may cost more.
Related: Input tokens, Output tokens, Long context pricing
Prompt caching
A cost optimization where repeated input tokens (system prompts, few-shot examples) are cached and charged at a lower rate on subsequent requests. Reduces input costs 75-90% for cached tokens.
Related: Cache hit rate, Cached input pricing, Cache write pricing
Cache hit rate
The percentage of input tokens that match cached content from previous requests. Higher hit rates mean lower effective input costs. Depends on prompt structure and request patterns.
Related: Prompt caching, Cache reuse models
Model Selection
Model routing
Automatically directing requests to the cheapest suitable model based on task type, complexity, or quality requirements. Can reduce costs 40-60%.
Related: Routing cascade, Budget tier, Model selection
Routing cascade
A model routing strategy that uses budget, mid, premium, and reasoning models in a tiered approach. The 60/30/10 rule: 60% budget, 30% mid, 10% premium+reasoning.
Related: Model routing, Budget tier, Premium tier
Budget tier
Models priced under $0.30 per 1M tokens. Fast and cheap for classification, extraction, simple Q&A, and high-volume tasks.
Related: Mid tier, Premium tier, Model routing
Mid tier
Models priced $0.30-$2.00 per 1M tokens. Balanced capability for RAG, summarization, and customer-facing chat.
Related: Budget tier, Premium tier, Model routing
Reasoning models
Models that use chain-of-thought reasoning for math, logic, multi-step planning, and self-critique. Typically priced higher due to increased compute per token.
Related: Premium tier, Thinking tokens, Reasoning premium
Capabilities
Vision
The ability to process image inputs alongside text. Vision models can analyze screenshots, photos, documents, and diagrams. Image inputs may have separate per-image pricing.
Related: Multimodal, Image input, PDF input
Multimodal
Models that can process multiple types of input (text, images, audio, video) and sometimes output multiple modalities. Vision and audio models are common examples.
Related: Vision, Audio input, Image generation
Function calling
The ability to call external tools and APIs during generation. Models output structured function calls that your application executes, enabling agentic workflows.
Related: Tool choice, Agentic AI, Parallel function calling
Agentic AI
AI systems that autonomously perform multi-step tasks using tools, memory, and planning. Requires function calling, context management, and cost estimation for repeated turns.
Related: Function calling, Tool choice, Context accumulation
Embedding
Converting text to numerical vectors for semantic search, retrieval, and similarity comparison. Embedding models are priced per token and used in RAG systems.
Related: Vector search, RAG, Semantic similarity
RAG
Retrieval-Augmented Generation. A pattern where relevant context is retrieved from a knowledge base and injected into the prompt before generation. Reduces hallucination and grounds responses.
Related: Embedding, Vector search, Context window
Fine-tuning & Training
Fine-tuning
Training a pre-trained model on your specific data to improve performance on targeted tasks. Involves training costs (per token) and ongoing inference markup.
Related: LoRA, SFT, Training cost, Inference markup
LoRA
Low-Rank Adaptation. A fine-tuning method that's cheaper than full fine-tuning but may produce lower quality for complex tasks. Typically 10-20% the cost of full fine-tuning.
Related: Fine-tuning, SFT, DPO
SFT
Supervised Fine-Tuning. Training a model on input-output pairs to learn a specific task or style. The standard fine-tuning approach for most use cases.
Related: Fine-tuning, LoRA, DPO
DPO
Direct Preference Optimization. A fine-tuning method that trains models to prefer certain outputs over others based on human feedback. Typically 10-20% more expensive than SFT.
Related: Fine-tuning, SFT, RLHF
Inference markup
The price increase for using a fine-tuned model compared to the base model. Varies by provider: Fireworks AI charges same-as-base, OpenAI charges 2-4x.
Related: Fine-tuning, Base model pricing, Hosting fees
Advanced Concepts
Context accumulation
The increasing cost of agentic workflows as context grows with each turn. Tool calls, results, and conversation history accumulate, requiring careful cost management.
Related: Agentic AI, Context window, Cache reuse
Output-to-input ratio
The relationship between output token pricing and input token pricing. Lower ratios mean output is cheaper relative to input, which benefits generation-heavy workloads.
Related: Output tokens, Input tokens, Lowest output ratio
Model identity
The unique identifier for a model route, combining provider, model family, and variant. Example: openai/gpt-4o-2024-08-06. Different providers may serve the same base model at different prices.
Related: Provider route, Model slug, Base model
Provider route
The specific path a model is served through. Same base model can have different routes (e.g., OpenAI direct vs Azure) with different pricing, limits, and features.
Related: Model identity, Base model, Pricing
Start exploring
Now that you understand the terms, use these tools to put them into practice: