TL;DR
Startups under 20 people should default to API-based deployment due to hidden operational costs, while enterprises with sustained volume and compliance needs should evaluate self-hosting for 40–70% lower per-token costs. Mid-sized teams typically benefit most from a hybrid routing strategy.
Conclusion
The choice between building and buying is not a one-time decision but a journey that evolves with your scale. Startups should prioritize speed through APIs, while enterprises must eventually weigh the operational burden of self-hosting against the significant savings and sovereignty it provides for high-volume, sensitive workloads.
You route different workloads to different infrastructure based on sensitivity, latency requirements, and volume. Common pattern: API for bursty, non-sensitive, variable workloads; self-hosted for steady-state, sensitive, or latency-critical workloads.
Best for: Most organisations above 50 people. Pure API or pure self-hosted is almost never optimal once you have a more complex mix of use cases with different constraints.
Real costs: Two infrastructure bills instead of one. But the total is usually lower than either extreme because you are not overpaying for API tokens on high-volume workloads nor over-provisioning GPUs for variable loads.
Decision framework
By team size
| Team size | Default approach | Why |
|---|---|---|
| 1-5 people | API only | No capacity for infrastructure work; self-hosting burns founder time more expensive than API markups |
| 5-20 people | API, evaluate self-hosting for 1-2 highest-volume workloads | One specialised workload may cross the breakeven threshold; everything else stays on API |
| 20-100 people | Hybrid with clear routing rules | You have enough throughput and headcount to justify selective self-hosting, but not full coverage |
| 100-500 people | Hybrid with dedicated infra team | Self-host the core product path; API for experiments, prototypes, and burst capacity |
| 500+ people | Self-hosted by default, API for specific niches | Data sovereignty, compliance, and cost optimisation drive toward self-hosting; API for niche model access |
By budget
Under $1,000/month in inference costs: APIs. Period. No GPU rental at this level is cheaper than API pricing once you include the human time to set up and maintain the serving stack.
$1,000-10,000/month: Evaluate self-hosting for the single highest-volume workload. Run the numbers: if >60% of your token spend is on one model and one task, the GPU breakeven is probably within reach.
$10,000-100,000/month: Hybrid. Self-host the 2-3 highest-volume workloads. Negotiate volume discounts on API usage for everything else. Most providers offer tiered pricing above $10K/month.
$100,000+/month: Full self-hosting evaluation with dedicated infrastructure team. At this spend level, the savings from self-hosting (40-70%) fund the infra team salary multiple times over.
By data sensitivity
| Sensitivity level | Data examples | Recommended approach |
|---|---|---|
| Public or internal-only | Public knowledge Q&A, code generation for open-source projects, marketing content drafting | API is fine. Normal data retention policies apply. |
| Business-sensitive | Financial projections, product roadmap planning, customer PII in prompts | API with enterprise data processing agreement (no-training clause, zero-data-retention). Or self-host if API provider’s terms are insufficient. |
| Regulated (PHI, financial records, legal privilege) | Patient health information, trade execution data, attorney-client communications | Self-hosted only. No API provider’s consumer or business tier provides sufficient guarantees for regulated data. Enterprise BAAs exist for healthcare but require dedicated infrastructure segments. |
| Classified or sovereign | Government data, defence, critical national infrastructure | Self-hosted on air-gapped or sovereignty-controlled infrastructure. No API option. |
By latency requirements
| Requirement | Viable approaches |
|---|---|
| Real-time interactive (<500ms P95) | API (if GPUs are nearby and provider has low P95); self-hosted on the same region/availability zone |
| Near-real-time (500ms-2s) | Either approach works; API is simpler |
| Batch/async (>2s acceptable) | Self-hosted on spot/preemptible GPUs for lowest cost; API for simplicity |
| Hard real-time (<100ms P99) | Self-hosted with dedicated GPU instances; API latency variance is too high |
Worked scenarios
Scenario A: The funded startup building a customer support summariser
Context: 8-person startup, $2M seed round, building a tool that summarises customer support tickets for small e-commerce businesses. 5,000 tickets/day, average prompt length 2,000 tokens, needs <2s response time.
Analysis: 5,000 tickets × 2,000 input tokens = 10M input tokens/day. On GPT-5 API, roughly $350-500/day. At a 30-person support team’s salary, this is justifiable, but the burn is real. A single H100 could handle this volume on a 7B model at roughly $70-100/day all-in (cloud GPU + storage). But the startup has zero ML ops experience.
Recommendation: API for now. The $250-400/day premium over self-hosting is the price of speed — the startup should be iterating on product, not debugging vLLM configs. Re-evaluate at 20M tokens/day or when a dedicated infrastructure hire is on the team.
Scenario B: The mid-market SaaS platform with compliance requirements
Context: 150-person company, SOC2 Type II, fintech data in prompts to generate compliance summaries. 50M tokens/day sustained, with spikes to 100M during end-of-month reporting. Latency-sensitive (needs <1s for user-facing feature). Cannot route fintech data through general API consumer endpoints.
Analysis: 50M tokens/day puts the API bill at roughly $1,500-2,500/day. A self-hosting setup with 2-4 H100s covers the baseline and absorbs the spikes with elastic cloud GPU backup. Estimated self-hosting cost: $300-500/day base + $100-200/day burst = $400-700/day. Savings: roughly $1,000-1,800/day. Infrastructure headcount: one senior MLOps engineer at $200K/year.
Recommendation: Self-host the core compliance workload on dedicated GPU instances with burst capacity. Keep the existing API connection for non-sensitive workloads (general Q&A, internal documentation assistant). The savings fund the infra hire within 3-5 months.
Scenario C: The enterprise deploying across multiple business units
Context: 3,000-person company, six business units each building LLM features independently. Combined inference spend $350K/month, growing 15% month-over-month. No central infrastructure team — each BU has its own API accounts, its and its own evaluation workflows, its own model choices.
Analysis: At this spend level, the self-hosting savings potential is $140-245K/month. But the real problem is fragmentation, not unit cost. Six teams independently evaluating models, negotiating with providers, and building deployment infrastructure means six times the duplicated effort.
Recommendation: Central infrastructure team running a shared GPU pool with per-BU tenant isolation. Self-host the common models (Mistral Large, Llama 4, DeepSeek V4) on the shared pool. Each BU gets an API-compatible endpoint with their own rate limits and monitoring. API for niche models and burst capacity. The central team’s cost is covered by the savings from eliminating the six independent infrastructure stacks.
What teams get wrong
Assuming self-hosting is always cheaper. It is not. Below 5-10M tokens/day, the GPU cost alone (even on spot instances) is higher than API pricing for most models. And that is before you add the human cost of setup and maintenance.
Assuming API pricing is the only cost. Latency variance, rate limits, model deprecations, and provider downtime all have costs that do not show up on the invoice. A provider that changes pricing or deprecates a model you depend on causes migration work that a self-hosted setup does not.
Building for flexibility you do not need yet. A multi-provider routing layer, fallback logic, and migration tooling add engineering months before you have enough traffic to justify them. Start with one provider, measure, then add complexity only when the data says you need it.
Ignoring the switching cost. Once your application’s prompt engineering, output parsing, and evaluation workflows are tuned to a specific model, switching providers costs weeks of re-tuning. The cost of switching is real, even if it is not on the API bill.
Self-hosting the wrong workload. Self-hosting a low-volume, latency-tolerant, non-sensitive workload saves almost nothing while adding operational burden. Self-hosting a high-volume, latency-critical, sensitive workload changes the enough economics. Pick the right workload first.
Related guides
**- Model gateways and routers: OpenRouter, LiteLLM and build-vs-buy questions
- AI vendor lock-in: model APIs, embeddings, vector stores and eval data
- Fallback design: what happens when the AI call fails
- Data leakage in LLM apps: logs, prompts, files and vendor retention
## Source list
**- Anthropic API pricing — https://www.anthimant.com/pricing (accessed 2026-05-30)
- OpenAI API pricing — https://openai.com/api/pricing/ (accessed 2026-05-30)
- Lambda Labs GPU cloud pricing — https://lambdalabs.com/service/gpu-cloud/pricing (accessed 2026-05-30)
- RunPod GPU cloud pricing — https://www.runpod.io/pricing (accessed 2026-05-30)
- Artificial Analysis — inference benchmarks and provider comparisons, https://artificialanalysis.ai/ (accessed 2026-05-30)
- Anthropic, “Building effective agents” — https://www.anthropic.com/engineering/building-effective-agents (published December 2024, accessed 2026-05-30)
## Methodology
- Data checked: 2026-05-30 - Sources consulted: Provider pricing pages (Anthintropic, OpenAI, Lambda Labs, RunPod), industry inference benchmarks (Artificial Analysis), published deployment guidance (Anthropic “Building effective agents” December 2024), and calibrated operator experience from live self-hosting deployments. - Assumptions: GPU cost estimates assume H100 (80GB) as the reference hardware. Breakeven calculations are calibrated for 7B-70B parameter models, which cover the most common self-hosting scenarios. Infrastructure headcount cost assumes US-based senior MLOps at approximately $200K/year fully loaded. API costs are based on published pricing; enterprise-negotiated rates are typically 15-30% lower. - Limitations: Pricing figures are May 2026 snapshots and shift when API providers or GPU cloud vendors change rates. Breakeven thresholds (5-15M tokens/day) vary significantly by model size: a 7B model runs on one GPU, a 400B MoE model needs 4-8 GPUs. Hardware assumptions use H100 only; A100, B200, Trainium, and TPU have different economics. Headcount cost is location-dependent — teams in lower-cost markets reach the breakeven point earlier. No provider-level negotiation pricing is reflected. - Jurisdiction: Global. Data sensitivity guidance references regulatory categories (SOC2, PHintropic, PHI/HIPMA, legal privilege) that apply across jurisdictions, but specific compliance obligations vary by country and industry.
Trust Stack
- Last checked: 2026-05-30
- Corrections: Contact us to report errors