LLMs for startups vs enterprises: when to build, when to buy

The most common question in LLM procurement is not which model to use. It is whether to call an API or run your own infrastructure.

The answer depends less on the model and more on who you are. A five-person startup and a five-hundred-person enterprise face different constraints on budget, latency tolerance, data sensitivity, headcount for ops, and willingness to lock into a provider. The same decision rule — “use APIs when you are small, self-host when you are big” — leads both teams to the wrong answer if applied mechanically.

This article lays out the decision framework by organisational context: when APIs make sense, when self-hosting pays off, and where the hybrid approaches that most serious teams end up with actually live.

Quick answer

Startups (under 20 people, no dedicated ML ops, constrained cash runway) should default to API-based deployment. The cost advantage of self-hosting only kicks in at sustained throughput levels that most early-stage startups do not reach. The hidden costs — GPU management, serving stack maintenance, capacity planning — are harder to absorb than API markups.

Enterprises (200+ people, dedicated infrastructure teams, compliance requirements, predictable or growing throughput) should evaluate self-hosting for any workload that runs at sustained volume. The per-token cost at scale is typically 40-70% lower than API pricing, and data sovereignty requirements often rule out third-party API providers for sensitive workloads.

Mid-sized teams (20-200 people) are where the decision is genuinely hard. The right answer is almost always hybrid: use APIs for variable-load, latency-tolerant, non-sensitive workloads and self-host for high-volume, latency-critical, or data-sensitive workloads.

But these are rules of thumb, not laws. The decision framework below lets you test your own numbers.

The three deployment models

API-based (buy)

You send prompts to a hosted API — OpenAI, Anthropic, Google, Mistral, Together, Fireworks — and pay per token. No GPUs to manage, no serving stack to maintain, no capacity to plan.

Best for: Low-to-moderate query volume, variable or unpredictable traffic, teams without dedicated infrastructure headcount, workloads that can tolerate third-party data handling.

Real costs: At 1M input tokens/day on GPT-5 or Claude 4, you are looking at roughly $35-65/day in input costs alone, with output tokens at roughly 3-4x that rate. At 10M tokens/day, the API bill exceeds $500/day on frontier models — enough to start asking whether self-hosting would be cheaper.

Hidden cost: Dependency. Once your application logic is built around a specific API’s schema, rate limits, and latency profile, switching providers is not free. Prompt engineering, tool-use patterns, and output parsing all embed assumptions about the specific model and API behaviour.

Self-hosted (build)

You run an inference engine — vLLM, SGLang, TensorRT-LLM, TGI — on your own GPU hardware, on-premises or in a cloud tenant. You pay for GPUs, storage, networking, and the people who keep it running.

Best for: Sustained high throughput (typically >50M tokens/day), workloads with data sovereignty or compliance requirements that rule out third-party APIs, applications that need predictable sub-100ms latency, and teams with at least one person who can diagnose a CUDA OOM at 2 AM.

Real costs: A single H100 (80 GB) runs roughly $2-4/hour on cloud rental or $25,000-35,000 purchased outright. At $3/hour, that is $2,160/month. At 50M tokens/day on a medium-sized model (7-13B), that GPU can handle the load comfortably — bringing per-token cost to roughly $1.44/M tokens, compared to $15-25/M tokens on the API. The breakeven against API pricing is typically between 5-15M tokens/day depending on the model size and hardware used.

Hidden cost: Operational burden. GPU instance provisioning, inference engine configuration, model downloads and storage, monitoring and alerting, security patching, scaling decisions, and the inevitable “why is throughput dropping?” debugging session all consume engineering time that could be spent on product features.

Hybrid

You route different workloads to different infrastructure based on sensitivity, latency requirements, and volume. Common pattern: API for bursty, non-sensitive, variable workloads; self-hosted for steady-state, sensitive, or latency-critical workloads.

Best for: Most organisations above 50 people. Pure API or pure self-hosted is almost never optimal once you have a mix of use cases with different constraints.

Real costs: Two infrastructure bills instead of one. But the total is usually lower than either extreme because you are not overpaying for API tokens on high-volume workloads nor over-provisioning GPUs for variable loads.

Decision framework

By team size

By budget

Under $1,000/month in inference costs: APIs. Period. No GPU rental at this level is cheaper than API pricing once you include the human time to set up and maintain the serving stack.

$1,000-10,000/month: Evaluate self-hosting for the single highest-volume workload. Run the numbers: if >60% of your token spend is on one model and one task, the GPU breakeven is probably within reach.

$10,000-100,000/month: Hybrid. Self-host the 2-3 highest-volume workloads. Negotiate volume discounts on API usage for everything else. Most providers offer tiered pricing above $10K/month.

$100,000+/month: Full self-hosting evaluation with dedicated infrastructure team. At this spend level, the savings from self-hosting (40-70%) fund the infra team salary multiple times over.

By data sensitivity

By latency requirements

Worked scenarios

Scenario A: The funded startup building a customer support summariser

Context: 8-person startup, $2M seed round, building a tool that summarises customer support tickets for small e-commerce businesses. 5,000 tickets/day, average prompt length 2,000 tokens, needs <2s response time.

Analysis: 5,000 tickets × 2,000 input tokens = 10M input tokens/day. On GPT-5 API, roughly $350-500/day. At a 30-person support team’s salary, this is justifiable, but the burn is real. A single H100 could handle this volume on a 7B model at roughly $70-100/day all-in (cloud GPU + storage). But the startup has zero ML ops experience.

Recommendation: API for now. The $250-400/day premium over self-hosting is the price of speed — the startup should be iterating on product, not debugging vLLM configs. Re-evaluate at 20M tokens/day or when a dedicated infrastructure hire is on the team.

Scenario B: The mid-market SaaS platform with compliance requirements

Context: 150-person company, SOC2 Type II, fintech data in prompts to generate compliance summaries. 50M tokens/day sustained, with spikes to 100M during end-of-month reporting. Latency-sensitive (needs <1s for user-facing feature). Cannot route fintech data through general API consumer endpoints.

Analysis: 50M tokens/day puts the API bill at roughly $1,500-2,500/day. A self-hosting setup with 2-4 H100s covers the baseline and absorbs the spikes with elastic cloud GPU backup. Estimated self-hosting cost: $300-500/day base + $100-200/day burst = $400-700/day. Savings: roughly $1,000-1,800/day. Infrastructure headcount: one senior MLOps engineer at $200K/year.

Recommendation: Self-host the core compliance workload on dedicated GPU instances with burst capacity. Keep the existing API connection for non-sensitive workloads (general Q&A, internal documentation assistant). The savings fund the infra hire within 3-5 months.

Scenario C: The enterprise deploying across multiple business units

Context: 3,000-person company, six business units each building LLM features independently. Combined inference spend $350K/month, growing 15% month-over-month. No central infrastructure team — each BU has its own API accounts, its own evaluation workflows, its own model choices.

Analysis: At this spend level, the self-hosting savings potential is $140-245K/month. But the real problem is fragmentation, not unit cost. Six teams independently evaluating models, negotiating with providers, and building deployment infrastructure means six times the duplicated effort.

Recommendation: Central infrastructure team running a shared GPU pool with per-BU tenant isolation. Self-host the common models (Mistral Large, Llama 4, DeepSeek V4) on the shared pool. Each BU gets an API-compatible endpoint with their own rate limits and monitoring. API for niche models and burst capacity. The central team’s cost is covered by the savings from eliminating the six independent infrastructure stacks.

What teams get wrong

Assuming self-hosting is always cheaper. It is not. Below 5-10M tokens/day, the GPU cost alone (even on spot instances) is higher than API pricing for most models. And that is before you add the human cost of setup and maintenance.

Assuming API pricing is the only cost. Latency variance, rate limits, model deprecations, and provider downtime all have costs that do not show up on the invoice. A provider that changes pricing or deprecates a model you depend on causes migration work that a self-hosted setup does not.

Building for flexibility you do not need yet. A multi-provider routing layer, fallback logic, and migration tooling add engineering months before you have enough traffic to justify them. Start with one provider, measure, then add complexity only when the data says you need it.

Ignoring the switching cost. Once your application’s prompt engineering, output parsing, and evaluation workflows are tuned to a specific model, switching providers costs weeks of re-tuning. The cost of switching is real, even if it is not on the API bill.

Self-hosting the wrong workload. Self-hosting a low-volume, latency-tolerant, non-sensitive workload saves almost nothing while adding operational burden. Self-hosting a high-volume, latency-critical, sensitive workload changes the economics. Pick the right workload first.

Related guides

• Model gateways and routers: OpenRouter, LiteLLM and build-vs-buy questions
• AI vendor lock-in: model APIs, embeddings, vector stores and eval data
• Fallback design: what happens when the AI call fails
• Data leakage in LLM apps: logs, prompts, files and vendor retention

Source list

• Anthropic API pricing — https://www.anthropic.com/pricing (accessed 2026-05-30)
• OpenAI API pricing — https://openai.com/api/pricing/ (accessed 2026-05-30)
• Lambda Labs GPU cloud pricing — https://lambdalabs.com/service/gpu-cloud/pricing (accessed 2026-05-30)
• RunPod GPU cloud pricing — https://www.runpod.io/pricing (accessed 2026-05-30)
• Artificial Analysis — inference benchmarks and provider comparisons, https://artificialanalysis.ai/ (accessed 2026-05-30)
• Anthropic, “Building effective agents” — https://www.anthropic.com/engineering/building-effective-agents (published December 2024, accessed 2026-05-30)

Methodology

• Data checked: 2026-05-30
• Sources consulted: Provider pricing pages (Anthropic, OpenAI, Lambda Labs, RunPod), industry inference benchmarks (Artificial Analysis), published deployment guidance (Anthropic “Building effective agents” December 2024), and calibrated operator experience from live self-hosting deployments.
• Assumptions: GPU cost estimates assume H100 (80GB) as the reference hardware. Breakeven calculations are calibrated for 7B-70B parameter models, which cover the most common self-hosting scenarios. Infrastructure headcount cost assumes US-based senior MLOps at approximately $200K/year fully loaded. API costs are based on published pricing; enterprise-negotiated rates are typically 15-30% lower.
• Limitations: Pricing figures are May 2026 snapshots and shift when API providers or GPU cloud vendors change rates. Breakeven thresholds (5-15M tokens/day) vary significantly by model size: a 7B model runs on one GPU, a 400B MoE model needs 4-8 GPUs. Hardware assumptions use H100 only; A100, B200, Trainium, and TPU have different economics. Headcount cost is location-dependent — teams in lower-cost markets reach the breakeven point earlier. No provider-level negotiation pricing is reflected.
• Jurisdiction: Global. Data sensitivity guidance references regulatory categories (SOC2, PHI/HIPAA, legal privilege) that apply across jurisdictions, but specific compliance obligations vary by country and industry.

Trust Stack

• Last substantive check: 2026-05-30
• Corrections policy: If you spot an error, contact us via the Contact page
• Affiliation: theLLMs has no vendor affiliation, sponsorship, or commercial relationship with any AI provider mentioned

Change log

• 2026-05-30: Editorial review against 16-gate checklist. Added 3 Editor’s Notes, Trust Stack, Change Log, replaced internal Sources table with URL-linked Source list, added Methodology section, trimmed description to 120-155 char range, set reviewedBy field.
• 2026-05-30: First published