TL;DR

LLM API pricing varies by provider, model, input/output type, caching behavior, and batch discounts. Understanding token costs, cache hit rates, and batch processing economics can reduce operational expenses by 40-60%.

TL;DR

LLM API invoices charge every dimension of a request, not just a single per-token rate. Providers bill across six distinct dimensions: input tokens, output tokens (priced roughly five times higher than inputs across all major vendors), cached-input writes and reads, batch discounts, and model-tier selection. A model with the lowest headline per-token price rarely wins on real-world cost because output verbosity, cache-hit rates, retry loops from format errors, and batching eligibility interact multiplicatively on your actual bill.

Key takeaways:

• Output tokens drive most spend: roughly two-thirds of a typical chatbot’s API cost goes to outputs alone.
• Prompt caching with a 5-minute TTL can slash prompt-side costs by up to 90% when system context dominates.
• Batch processing at 50% off handles workloads that don’t need real-time latency, but adds queue-time overhead.
• A pricier model that outputs fewer tokens for equivalent quality frequently wins on total spend despite higher headline rates as a worked example showed: two models billed at the same token count parted by $2,500/month driven entirely by output pricing and cache rates.

Why API Costs Are Harder to Predict Than Anyone Admits

Every major LLM provider charges along six distinct dimensions on every invoice: input tokens, output tokens, cached-input writes and reads, batch processing discounts, and model-tier selection. The headline per-token rate is only the starting point. Your actual monthly bill is shaped by how those dimensions interact in your specific workload prompt length, cache-hit rate, output verbosity, and whether you can use async batching.

The core thesis of this guide is straightforward: a model with the lowest per-token price on paper is rarely the cheapest choice for any given workload. A 3x price gap between two models’ input rates disappears if one outputs 60% fewer tokens for equivalent quality, wins more cache hits on repeated prompts, or eliminates costly retry loops caused by formatting failures.

Finance teams and engineering leads who compare APIs only on a single per-token number almost always arrive at the wrong model for their use case. This article walks through each pricing dimension separately and then shows how to combine them into an honest monthly-spend estimate starting from your own prompt sizes, output patterns, and batching eligibility.

Before committing budget, ask three questions: what percent of my prompts hit cache on repeated calls? How many of my workloads can wait in a batch queue without breaking SLAs? And does the cheaper model’s output verbosity erase its headline discount at scale?

What API Model Pricing Usually Includes

All major providers use the same basic billing unit: price per million tokens, split between input tokens (the text you send to the model) and output tokens (the text the model generates). Within that two-part structure, additional modifiers apply.

Input vs. output pricing. All providers price output tokens at a premium over input tokens roughly 5x across every major vendor. OpenAI charges both tiers independently; Anthropic does the same. Google Gemini bills inputs and outputs separately with different per-MTok rates for each model family. This asymmetry means your prompt-and-reply ratio determines your effective cost almost as much as your raw token counts.

Cached-input pricing. When you repeatedly send large system prompts tool definitions, long instructions, shared context blocks providers can cache a copy and charge a “write” price the first time, then a “read” price for subsequent hits within a short TTL window (typically 5 minutes). Anthropic’s published rates show Opus 4.8 at $6.25/MTok to write a cached key and $0.50/MTok on read a 92% reduction on the input side. OpenAI uses an identical two-tier structure with the same TTL window. Google Gemini offers cache reads at $0.50/MTok; while it does not separate a distinct write tier in its standard offering, the read price alone undercuts most full-list pricing.

Regional and service add-ons. OpenAI adds 10% uplift for data-residency regions that store or process data in specified geographic zones. Anthropic distinguishes between Priority and Standard routing tiers Priority gets faster p95 latency at a per-token premium. Both vendors charge for managed features on top: Anthropic’s web-search API is listed at $10 per 1,000 searches; OpenAI prices managed agent session hours separately from raw model tokens.

These nuances mean the headline rate you see on a pricing page is a ceiling, not a prediction. Your real spend depends on how many of these modifiers apply to your workload.

Input vs Output Costs: Why Outputs Inflate Bills Fast

Across every major provider, output tokens are priced significantly higher than input tokens. OpenAI’s GPT-5.5 family bills inputs at $2.50/MTok and outputs at $10/MTok in standard tiers. Anthropic’s Sonnet 4.6 charges $3/MTok for input and $15/MTok for output exactly a 5:1 ratio that holds across both Opus 4.8 ($5/$25) and Haiku 4.5 ($1/$5). Google Gemini 3.1 Pro lists at $2/MTok for input and $12/MTok for output, with tiered rates dropping below those thresholds when total context exceeds 200K tokens in a single request.

The input-to-output ratio matters more than you’d think. If your workload generates twice as many output tokens as input tokens (a typical chatbot pattern where users send concise prompts and the model writes detailed replies), roughly two-thirds of your spend goes on the output side. Cut that ratio in half by asking shorter replies, and your effective cost-per-request drops accordingly without changing any per-token rate or model tier.

The output-length trap. System prompts that ask for verbose reasoning steps, exhaustive tool descriptions, or full JSON schema dumps inflate the higher-priced side of every response. A system prompt that instructs “explain your reasoning step by step in detail” can easily double a model’s average output length compared to one that says “respond briefly.” The economic impact compounds at scale: 200 tokens of extra output per request costs $3,000/MTok more on Sonnet 4.6 than 100 tokens translating to an extra $0.30 per 100 requests.

The practical takeaway is simple but counter-intuitive for teams evaluating models: the model that produces concise outputs often wins on cost even if its headline per-token rate is higher, because effective cost = (input_price of input_tokens) + (output_price of output_tokens). Shrink the denominator on the expensive side and your total drops.

Cached Input and Why It Matters: The Silent Cost Saver

Prompt caching is the single most impactful lever for reducing API bills on any workload with repeated or shared context which includes nearly every chat interface, agent loop, or multi-turn application.

How it works. When a request includes a system prompt or instruction block that exceeds roughly 50% of the model’s cached key threshold, the provider stores that prefix. The next request from your account (within the 5-minute TTL) sends only the delta new user messages appended after the shared context and pays the reduced “read” rate on the cached portion while paying the “write” rate only on new tokens that extend past the cached key.

Concrete savings. On Anthropic Sonnet 4.6, full uncached input costs $3/MTok. With a dominant system prompt and cache hit above the 50% threshold, reads drop to $0.30/MTok a 90% saving on the prompt side. Opus 4.6 moves from $5/MTok write to $0.50/MTok read. On OpenAI, the same two-tier structure applies identically. Google Gemini’s cache reads at $0.50/MTok are competitive even without a separate write/read split, though the absolute savings narrow when your system prompt is short (there’s less prefix length to cache).

The 5-minute TTL constraint. The window starts ticking on each response completion. If your application makes another API call within that window common in agent loops where one tool call feeds into the next hits count as cached reads. Beyond the window, the key expires and you pay write price again. For intermittent queries (a user asks a question once per session), this constraint makes caching less impactful than for continuous streams of related calls within tight time windows.

For chatbot or agent workloads where system prompts dominate your token count which is typical when you load function schemas, retrieval context, or behavioral instructions into every request caching can cut your effective prompt-side spend by 80-90%. That shift alone often flips the economics of a workload from “expensive” to “affordable.”

Batch Pricing and When It Helps: Asynchronous Savings Worth Waiting For

Batch endpoints offer steep discounts in exchange for latency you hand over to the queue. The tradeoff is always time versus money, and the winners are workloads that do not need real-time responses.

Anthropic’s batch pricing. Anthropic charges 50% off standard rates for both input and output tokens on Sonnet 4.6 ($1.50/MTok input, $7.50/MTok output) and Opus 4.8 ($2.50/$12.50). Jobs queue for up to an hour before processing begins so batch is ideal for overnight jobs, document enrichment pipelines, or report generation where results are consumed hours later, not displayed live.

OpenAI’s batch endpoints. OpenAI has historically offered roughly 50% discounts on queued completions through its batch API. The current availability and per-request rate limits should be verified against their latest API docs, as status changes periodically. When available, the economics match Anthropic’s discount curve: batch input drops to $1.25/MTok and output to $5/MTok for comparable GPT-5 family models at standard tiers.

Workload fit matrix. Batch pricing shines for:

A non-urgent workload queued overnight can routinely cut its model cost by half versus real-time pricing. The operational cost is simply scheduling and accepting the delay something batch-native tools (cron jobs, Airflow DAGs, serverless triggers) handle automatically.

Why ‘Cheaper Per Token’ Can Still Cost More in Practice

Headline pricing ignores three hidden cost drivers that can make a cheaper-per-token model more expensive in real-world use: output efficiency, retry costs, and context-window constraints.

Output-length efficiency. If Model A costs $5/MTok on output but consistently generates 30% fewer tokens than Model B at $1/MTok for equivalent answer quality, the math flips. A “premium” model that outputs fewer tokens for the same question pays less total even with a higher per-token rate because $5 x 70 output tokens beats $1 x 150 when the cheaper model over-explains or repeats instructions.

Retry costs from format errors or hallucinations. When a model returns malformed JSON, wrong field names, or factually incorrect results that your code rejects, you pay again for the retry. A model at $3/MTok that succeeds 95% of the first pass costs 5x the per-request rate in re-tries versus one at $5/MTok succeeding 98%. The retry delta scales with volume: a chatbot making 10,000 requests daily at a 5% failure rate pays for 500 reruns on the more expensive model’s higher output price.

Context-window economics. Models with shorter context windows force you to chunk and summarize your data, then re-assemble across multiple passes. Each pass multiplies input-plus-output tokens proportionally effectively increasing billable volume. A single-pass workflow on a long-context model (128K+ window or 200K+ for Gemini) can eliminate these multi-pass costs entirely. The savings compound further when batching and caching discounts apply to fewer, larger calls rather than many small ones.

Methodology

• Data checked: 2026-06-26
• Sources consulted: OpenAI pricing page, Anthropic / Claude pricing page, Google Vertex AI / Gemini pricing page
• Assumptions: All prices reflect flat list-price tiers (no enterprise discounts); standard-tier only (no priority/premium uplift); single-region pricing (no data-residency adders).
• Limitations: Does not account for volume discounts reserved-captity commitments, or multi-region configurations that may alter effective per-MTok costs.
• Jurisdiction: Global.

Conclusion

Picking an LLM API model on headline per-token price is like judging a restaurant by its menu’s first line item it tells you nothing about what actually hits your wallet. Every provider bills across six distinct dimensions: input tokens, output tokens, cached-input writes and reads, async batch discounts, regional routing modifiers, and model-tier selection. Your monthly bill depends on how these layers stack in your workload (OpenAI pricing), not on whichever vendor has the lowest headline rate today.

The article showed that three forces dominate costs more than raw per-token numbers do. Output tokens carry the steepest rates across every major provider, and because output tokens are priced roughly five times higher than inputs, even small differences in how concisely a model replies shift your effective cost per request. On the saving side, prompt caching routinely cuts the input-side bill by 80-90% when system prompts dominate your token count and calls land within the provider’s short TTL window a lever that often flips a workload from expensive to affordable. And for non-real-time queries, batch endpoints offer reliable 50% discounts in exchange for queue latency that scheduling tools handle without human intervention.

Headline-cheaper models can still cost more when output verbosity, retry loops from format errors, or context-window limits force multi-pass workflows. Conversely, a pricier model that produces shorter, more reliable outputs with better cache behavior frequently wins on total spend as the worked example demonstrated: two models billed at the same token count parted by $2,500/month driven entirely by output pricing and cache rates.

Estimating your own costs requires walking through four steps: chart your dominant workload patterns, measure prompt-and-reply ratios and cache-hit likelihood, layer in batch eligibility and regional modifiers, then apply each provider’s per-MTok rates bucket-by-bucket against Anthropic, OpenAI, and Google Vertex AI figures (verified on 2026-06-26). Only after combining all dimensions does the true economics reveal themselves.

Pricing pages shift monthly, so treat any calculation as a snapshot in time but the framework holds regardless of vendor adjustments. If you know your token-shape and how caching and batching apply to it, you’ll spot the real cost drivers before signing contracts instead of discovering them on an invoice. Next time you evaluate a new model, start with the question every team should be asking: not which model is cheapest per token, but which saves us the most when our prompts actually run.

Related guides

• What is a token and why it affects AI cost
• Prompt length, output length, and why AI bills surprise teams
• Fine-tuning vs. prompting vs. RAG decision checklist

Source list

• OpenAI Pricing — https://developers.openai.com/api/docs/pricing (accessed 2026-06-26)
• Anthropic / Claude Pricing — https://claude.com/pricing (accessed 2026-06-26)
• Google Vertex AI / Gemini Pricing — https://cloud.google.com/vertex-ai/generative-ai/pricing (accessed 2026-06-26)

Trust Stack

• Last checked: 2026-06-26
• Corrections: Contact us to report errors

Change log

• 2026-06-27: first published