LLM inference cost per query: real-world estimator with worked examples

TL;DR

A single LLM API call costs anywhere from 0.002 cents to over 10 cents depending on the provider, model tier, and token counts involved. The three most common operator workloads — summarisation, RAG queries, and batch classification — typically land at very different price points:

The biggest cost driver is rarely the model itself. It is the context window length, cache-hit rate, and output verbosity — three factors most teams treat as afterthoughts.

The three-factor cost formula

Every LLM API call follows the same billing formula:

Cost = (input_tokens × input_price) + (output_tokens × output_price)

The provider charges you two separate line items: tokens you send in (prompt + system instructions + context) and tokens the model sends back. Input tokens are cheaper; output tokens are 3× to 10× more expensive per token.

But the formula hides three compounding variables that operators routinely underestimate:

1. Context window size — a 4K prompt costs 4× a 1K prompt per query
2. Cache hit rate — cached input can be 80–98% cheaper than uncached input, but only if the provider supports prompt caching and your workload reuses text
3. Output verbosity — a model that generates 500 tokens instead of 150 tokens multiplies your output cost by 3.3×

Worked example 1: text summarisation

Scenario: You run a customer support summarisation pipeline. Each ticket is 3,500 words of conversation history. Your system prompt adds 500 words of instructions and format constraints.

Token breakdown: ~4,000 input tokens + ~500 output tokens per ticket.

Provider costs per summarisation query (cents)

Using current per-million-token rates (checked 2026-05-30), converted to cents (¢) per query:

*DeepSeek v4 Pro at current 75% off promotional pricing. Post-promotion (after 2026-05-31 15:59 UTC): ~4× these rates.

At volume (10,000 tickets/day):

• Cheapest (Groq Llama 4 Scout): ~$6/day
• Mid-range (DeepSeek v4 Flash): $7/day
• Premium (Claude Sonnet 4.5): $195/day

Worked example 2: RAG query

Scenario: Your RAG application receives a user question, retrieves 8 highly relevant document chunks (~8,000 tokens of context), prepends a system prompt (~1,000 tokens), and generates a synthesised answer (~1,000 tokens).

Token breakdown: 1,000 (prompt + instructions) + 8,000 (retrieved chunks) + 1,000 (output) = 9,000 input / 1,000 output per query.

Provider costs per RAG query (cents)

*DeepSeek v4 Pro promotional pricing — see summarisation table for caveat.

With cache hits: If the system prompt and a shared knowledge base prefix (~5,000 tokens) are cached, the uncached input drops from 9,000 to 4,000 tokens. With DeepSeek’s 98% cache discount ($0.0028/M vs $0.14/M), the input cost for the cached portion falls to just 0.014¢ — reducing per-query cost by 80–85%.

Worked example 3: batch classification

Scenario: You need to classify 10,000 short text inputs (customer messages, moderation flags, routing labels). Each input is ~100 tokens. You send them as individual API calls with a shared system prompt (~500 tokens).

Token breakdown: 500 (system prompt) + 100 (input) = 600 input / ~20 output per item. 10,000 items total.

Provider costs for 10,000 classifications (total, cents)

*DeepSeek v4 Pro promotional pricing — see summarisation table for caveat.

With batch API discount (50% off for eligible workloads): OpenAI drops to ~$3.50 total; Google to $0.51; DeepSeek to $0.45 (Flash) / $1.39 (Pro).

At these volumes, caching matters less because the per-item token count is small. The dominant factor becomes the base input rate — which is why cost-conscious teams route high-volume classification to budget-tier models.

How context window size compounds cost

Context window size is the single most underestimated cost multiplier. A 32K-context RAG query costs 8× more in input tokens than a 4K summarisation task on the same model. The cost difference between an empty prompt and a 128K document-processing prompt is 128× before you generate a single output token.

Some providers add pricing cliffs at specific context thresholds:

• Google Gemini 2.5 Pro: Input costs double from $1.25/M to $2.50/M above 200K tokens. Output costs jump from $10/M to $15/M.
• Anthropic Claude models: The 200K context window has no cliff within size, but longer prompts mean more tokens at the standard input rate — a 200K prompt costs 200× a 1K prompt.
• DeepSeek: Flat pricing across the full 1M context window — no cliffs, but the absolute token count at 1M makes input the dominant cost term for any full-context query.

The operator rule: If your workload uses fewer than 10% of the context tokens available, reduce the context window size. Sending a 128K context when you only need 8K of retrieved chunks is a 16× cost multiplier for no quality gain.

How output verbosity multiplies real cost

Output tokens cost 3–10× more than input tokens per token. This asymmetry means the output length is the highest-leverage variable in your unit economics — more than model selection or provider choice.

Real example: A premium model that produces a 200-token summary at $15/M output costs 0.3¢ in output per query. A budget model that produces a 1,000-token rambling version at $0.28/M output costs only 0.028¢ in absolute terms — but if the budget model’s verbosity forces you to write a follow-up extraction step (another LLM call), the total cost can exceed the premium model.

Methodology

• Data checked: 2026-05-30
• Sources consulted: DeepSeek API docs, Google Vertex AI pricing, Anthropic Claude pricing, Groq pricing, Together AI pricing, theLLMs cross-provider pricing comparison, OpenAI pricing (estimated via third-party aggregator due to server-side request blocking)
• Assumptions: All currency values are USD. “M” means million tokens. ”¢” means US cents. Per-query costs are rounded to three decimal places where appropriate (sub-tenth-of-a-cent figures shown with a ”~” prefix). Cache-hit rates assume optimal configurations. Batch discounts shown where applicable. Output token estimates are typical for each workload type — actual output lengths vary by model and prompt.
• Limitations: This article provides per-query estimates based on typical token counts, not exact production costs. Actual costs vary with prompt engineering choices, model-specific output verbosity, retry rates, authentication overhead, network latency, rate-limit surcharges, and enterprise/EULA pricing tiers. It does not cover self-hosted inference costs, fine-tuning economics, enterprise volume discounts, reserved-capacity pricing, or currency/regional pricing differences. It does not include embedding costs for vector databases, storage costs, or the cost of running a reranker/retrieval pipeline alongside the LLM. It is not financial or procurement advice.
• Jurisdiction: Global. All pricing referenced from publicly available API documentation.

Source list

• DeepSeek models and pricing — accessed 2026-05-30
• Google Vertex AI pricing — accessed 2026-05-30
• Anthropic Claude pricing — accessed 2026-05-30
• Groq pricing — accessed 2026-05-30
• Together AI pricing — accessed 2026-05-30
• LLM API pricing comparison (theLLMs) — cross-provider reference data checked 2026-05-29
• Artificial Analysis model comparison — accessed 2026-05-30 (OpenAI rate estimates)

Related guides

• LLM API pricing comparison: GPT-5, Claude, Gemini, DeepSeek and Llama (2026)
• What is a token, and why does it affect AI cost?
• Prompt caching explained: when repeated context becomes cheaper
• The hidden cost of retries, fallbacks and validation loops
• Hosted API vs self-hosted open model: the real cost comparison

Trust Stack

• Last checked: 2026-05-30
• Corrections: Contact us to report errors

Change log

• 2026-05-30: Published. Three worked examples (summarisation, RAG, batch classification) with per-query cost tables across DeepSeek, Google, Anthropic, Groq and OpenAI. Sections on context-window cost compounding, output-verbosity multipliers, and prompt-caching economics. Provider pricing verified against official docs and cross-provider comparison.