theLLMs

Last checked: 2026-05-30

Scope: Global. Provider pricing checked 2026-05-30 from official docs and the site's cross-provider comparison. Pricing changes frequently — verify current rates before procurement decisions.

Key terms:

>_ large language model (LLM) >_ inference >_ LLM-as-a-judge >_ RAG (Retrieval-Augmented Generation) >_ batch API >_ HIPAA (Health Insurance Portability and Accountability Act)
Hero image for LLM inference cost per query: real-world estimator with worked examples

LLM inference cost per query: real-world estimator with worked examples ## TL;DR A single LLM API call costs anywhere from 0.002 cents to over 10 cents depending on the provider, model tier, and token counts involved. The three most common operator workloads — summarisation, RAG queries, and batch classification — typically land at very different price points: | Workload | Typical cost range (per query) |

| | | | Summarise a support ticket (4k in / 500 out) | $0.0006 – $0.021 | | RAG query with 8k context (1k+8k+1k) | $0.001 – $0.045 | | Batch classify 10,000 short inputs | $0.14 – $2.10 (total, not per-input) | The biggest cost driver is rarely the model itself. It is the context window length, cache-hit rate, and output verbosity — three factors most teams treat as afterthoughts. ## The three-factor cost formula Every LLM API call follows the same billing formula: ``` Cost = (input_tokens × input_price) + (output_tokens × output_price)

2. **Cache hit rate** — cached input can be 80–98% cheaper than uncached input, but only if the provider supports prompt caching and your workload reuses text
3. **Output verbosity** — a model that generates 500 tokens instead of 150 tokens multiplies your output cost by 3.3× <aside class="editor-note">
<strong>Editor's Note:</strong> Most teams model their LLM costs by multiplying provider headline rates by estimated monthly volumes. The result is almost always too low, because they assume (a) short outputs, (b) zero cache misses, and (c) fixed prompt sizes. Real-world costs for production systems are typically 1.5–3× higher than these back-of-envelope estimates. The gap comes from retries, validation loops, longer-than-expected outputs, and prompt drift.
</aside> ## Worked example 1: text summarisation **Scenario:** You run a customer support summarisation pipeline. Each ticket is 3,500 words of conversation history. Your system prompt adds 500 words of instructions and format constraints. **Token breakdown:** ~4,000 input tokens + ~500 output tokens per ticket. ### Provider costs per summarisation query (cents) Using current per-million-token rates (checked 2026-05-30), converted to cents (¢) per query: | Provider | Model | Input cost (4k) | Output cost (500) | Total per query |
| | | | | |
| DeepSeek | v4 Flash | 0.056¢ | 0.014¢ | **0.070¢** |
| DeepSeek | v4 Pro* | 0.174¢ | 0.044¢ | **0.218¢** |
| Google | Gemini 2.0 Flash | 0.060¢ | 0.030¢ | **0.090¢** |
| Google | Gemini 2.5 Pro | 0.500¢ | 0.500¢ | **1.000¢** |
| Anthropic | Claude Sonnet 4.5 | 1.200¢ | 0.750¢ | **1.950¢** |
| Groq | Llama 4 Scout | ~0.040¢ | ~0.020¢ | **~0.060¢** |
| OpenAI (est.) | GPT-5 mid | ~0.800¢ | ~0.500¢ | **~1.300¢** | \*DeepSeek v4 Pro at current 75% off promotional pricing. Post-promotion (after 2026-05-31 15:59 UTC): ~4× these rates. **At volume** (10,000 tickets/day):
- Cheapest (Groq Llama 4 Scout): **~$6/day**
- Mid-range (DeepSeek v4 Flash): **$7/day**
- Premium (Claude Sonnet 4.5): **$195/day** <aside class="editor-note">
<strong>Editor's Note:</strong> The summary above uses fixed output lengths — 500 tokens. In practice, a budget model may produce 800 tokens of rambling output to express the same information that a premium model captures in 200 tokens. That 4× output multiplier on a cheap model can easily erase the headline input-rate advantage. Always measure the actual output token distribution in production before fixing your cost model.
</aside> ## Worked example 2: RAG query **Scenario:** Your RAG application receives a user question, retrieves 8 highly relevant document chunks (~8,000 tokens of context), prepends a system prompt (~1,000 tokens), and generates a synthesised answer (~1,000 tokens). **Token breakdown:** 1,000 (prompt + instructions) + 8,000 (retrieved chunks) + 1,000 (output) = **9,000 input / 1,000 output** per query. ### Provider costs per RAG query (cents) | Provider | Model | Input cost (9k) | Output cost (1k) | Total per query |
| | | | | |
| DeepSeek | v4 Flash | 0.126¢ | 0.028¢ | **0.154¢** |
| DeepSeek | v4 Pro* | 0.392¢ | 0.087¢ | **0.479¢** |
| Google | Gemini 2.0 Flash | 0.135¢ | 0.060¢ | **0.195¢** |
| Google | Gemini 2.5 Pro | 1.125¢ | 1.000¢ | **2.125¢** |
| Anthropic | Claude Sonnet 4.5 | 2.700¢ | 1.500¢ | **4.200¢** |
| Groq | Llama 4 Scout | ~0.090¢ | ~0.040¢ | **~0.130¢** |
| OpenAI (est.) | GPT-5 mid | ~1.800¢ | ~1.000¢ | **~2.800¢** | \*DeepSeek v4 Pro promotional pricing — see summarisation table for caveat. **With cache hits:** If the system prompt and a shared knowledge base prefix (~5,000 tokens) are cached, the uncached input drops from 9,000 to 4,000 tokens. With DeepSeek's 98% cache discount ($0.0028/M vs $0.14/M), the input cost for the cached portion falls to just 0.014¢ — reducing per-query cost by 80–85%. <aside class="editor-note">
<strong>Editor's Note:</strong> Prompt caching transforms RAG economics, but it only pays off when the same context is reused across queries. A RAG system where every query retrieves different document chunks gets negligible cache benefit on the chunk tokens — only the static system prompt and instructions are reusable. The highest-value caching strategy is to put your longest fixed text (brand guidelines, format constraints, guardrails) into the cacheable prefix. The chunks themselves almost never cache.
</aside> ## Worked example 3: batch classification **Scenario:** You need to classify 10,000 short text inputs (customer messages, moderation flags, routing labels). Each input is ~100 tokens. You send them as individual API calls with a shared system prompt (~500 tokens). **Token breakdown:** 500 (system prompt) + 100 (input) = **600 input / ~20 output** per item. 10,000 items total. ### Provider costs for 10,000 classifications (total, cents) | Provider | Model | Input cost (600 × 10k) | Output cost (20 × 10k) | Total batch cost |
| | | | | |
| DeepSeek | v4 Flash | $0.84 (0.14¢ × 6M) | $0.056 (0.28¢ × 0.2M) | **$0.90** |
| DeepSeek | v4 Pro* | $2.61 | $0.17 | **$2.78** |
| Google | Gemini 2.0 Flash | $0.90 | $0.12 | **$1.02** |
| Anthropic | Claude Sonnet 4.5 | $18.00 | $3.00 | **$21.00** |
| Groq | Llama 4 Scout | ~$0.60 | ~$0.08 | **~$0.68** |
| OpenAI (est., batch 50% off) | GPT-5 mid | ~$6.00 | ~$1.00 | **~$7.00** | \*DeepSeek v4 Pro promotional pricing — see summarisation table for caveat. **With batch API discount** (50% off for eligible workloads): OpenAI drops to ~$3.50 total; Google to $0.51; DeepSeek to $0.45 (Flash) / $1.39 (Pro). At these volumes, **caching matters less** because the per-item token count is small. The dominant factor becomes the **base input rate** — which is why cost-conscious teams route high-volume classification to budget-tier models. ## How context window size compounds cost Context window size is the single most underestimated cost multiplier. A 32K-context RAG query costs **8× more in input tokens** than a 4K summarisation task on the same model. The cost difference between an empty prompt and a 128K document-processing prompt is 128× before you generate a single output token. Some providers add pricing cliffs at specific context thresholds: - **Google Gemini 2.5 Pro:** Input costs double from $1.25/M to $2.50/M above 200K tokens. Output costs jump from $10/M to $15/M.
- **Anthropic Claude models:** The 200K context window has no cliff within size, but longer prompts mean more tokens at the standard input rate — a 200K prompt costs 200× a 1K prompt.
- **DeepSeek:** Flat pricing across the full 1M context window — no cliffs, but the absolute token count at 1M makes input the dominant cost term for any full-context query. **The operator rule:** If your workload uses fewer than 10% of the context tokens available, reduce the context window size. Sending a 128K context when you only need 8K of retrieved chunks is a 16× cost multiplier for no quality gain. ## How output verbosity multiplies real cost Output tokens cost 3–10× more than input tokens per token. This asymmetry means the output length is the highest-leverage variable in your unit economics — more than model selection or provider choice. **Real example:** A premium model that produces a 200-token summary at $15/M output costs 0.3¢ in output per query. A budget model that produces a 1,000-token rambling version at $0.28/M output costs only 0.028¢ in absolute terms — but if the budget model's verbosity forces you to write a follow-up extraction step (another LLM call), the total cost can exceed the premium model. <aside class="editor-note">
<strong>Editor's Note:</strong> The cheapest model is rarely the cheapest in production. Budget models tend to produce longer, less structured output — which often requires validation retries, output parsing, or a second pass. A model that costs 10× less per token but produces 3× more output and requires 2× more retries ends up costing more, not less. Measure the total pipeline cost, not the per-token rate.
</aside> ## Methodology - **Data checked:** 2026-05-30
- **Sources consulted:** DeepSeek API docs, Google Vertex AI pricing, Anthropic Claude pricing, Groq pricing, Together AI pricing, theLLMs cross-provider pricing comparison, OpenAI pricing (estimated via third-party aggregator due to server-side request blocking)
- **Assumptions:** All currency values are USD. "M" means million tokens. "¢" means US cents. Per-query costs are rounded to three decimal places where appropriate (sub-tenth-of-a-cent figures shown with a "~" prefix). Cache-hit rates assume optimal configurations. Batch discounts shown where applicable. Output token estimates are typical for each workload type — actual output lengths vary by model and prompt.
- **Limitations:** This article provides per-query estimates based on typical token counts, not exact production costs. Actual costs vary with prompt engineering choices, model-specific output verbosity, retry rates, authentication overhead, network latency, rate-limit surcharges, and enterprise/EULA pricing tiers. It does not cover self-hosted inference costs, fine-tuning economics, enterprise volume discounts, reserved-capacity pricing, or currency/regional pricing differences. It does not include embedding costs for vector databases, storage costs, or the cost of running a reranker/retrieval pipeline alongside the LLM. It is not financial or procurement advice.
- **Jurisdiction:** Global. All pricing referenced from publicly available API documentation. ## Source list - [DeepSeek models and pricing](https://api-docs.deepseek.com/quick_start/pricing) — accessed 2026-05-30
- [Google Vertex AI pricing](https://cloud.google.com/vertex-ai/generative-ai/pricing) — accessed 2026-05-30
- [Anthropic Claude pricing](https://www.anthropic.com/pricing) — accessed 2026-05-30
- [Groq pricing](https://groq.com/pricing/) — accessed 2026-05-30
- [Together AI pricing](https://www.together.ai/pricing) — accessed 2026-05-30
- [LLM API pricing comparison (theLLMs)](/cache/llm-api-pricing-comparison-2026/) — cross-provider reference data checked 2026-05-29
- [Artificial Analysis model comparison](https://artificialanalysis.ai/models) — accessed 2026-05-30 (OpenAI rate estimates) ## Related guides - [LLM API pricing comparison: GPT-5, Claude, Gemini, DeepSeek and Llama (2026)](/cache/llm-api-pricing-comparison-2026/)
- [What is a token, and why does it affect AI cost?](/cache/what-is-a-token-and-why-it-affects-ai-cost/)
- [Prompt caching explained: when repeated context becomes cheaper](/cache/prompt-caching-explained-when-repeated-context-becomes-cheaper/)
- [The hidden cost of retries, fallbacks and validation loops](/cache/the-hidden-cost-of-retries-fallbacks-and-validation-loops/)
- [Hosted API vs self-hosted open model: the real cost comparison](/comparisons/hosted-api-vs-self-hosted-open-model/) ## Trust Stack - **Last checked:** 2026-05-30
- **Corrections:** [Contact us](/contact/) to report errors ## Change log - **2026-05-30:** Published. Three worked examples (summarisation, RAG, batch classification) with per-query cost tables across DeepSeek, Google, Anthropic, Groq and OpenAI. Sections on context-window cost compounding, output-verbosity multipliers, and prompt-caching economics. Provider pricing verified against official docs and cross-provider comparison.