theLLMs

Last checked: 2026-05-25

Scope: Global. Provider fine-tuning and training docs were checked on 2026-05-25; commercial and technical terms vary by provider. Data residency and provider data-use policies affect training and fine-tuning workflow choices — see regional caveats below.

AI draft model: gpt-5.4-mini

AI review model: llm-editor (deepseek-v4-pro)

Key terms:

>_ large language model (LLM) >_ inference >_ training >_ pre-training >_ fine-tuning >_ SFT (Supervised Fine-Tuning)

Inference vs training vs fine-tuning: three terms operators confuse

These three terms are easy to mix up because they all involve a model, data and money. But they are different jobs.

  • Inference is using a model to answer a request.
  • Training is changing the model’s weights from scratch or with a large optimisation run.
  • Fine-tuning is adapting an existing model to a narrower task or style.

If you blur those together, you can pick the wrong budget, the wrong data process and the wrong risk controls.

“We should fine-tune it” is often a vague wish, not a technical decision [1]. In many products, inference plus better prompting or retrieval is the first sensible move. Fine-tuning is useful, but it is not the default answer to every mismatch.

Quick answer

If you need a live feature to respond to users, you are doing inference.

If you need to teach a model new behaviour using a curated dataset, you may be fine-tuning.

If you are building a model family or doing heavy pretraining work, you are in training territory, which is a different scale of cost, data and expertise.

What each term means

Inference

Inference is the moment the model is used.

The model already exists, and you send it a prompt or other input. The system then produces an output. Most business LLM products spend most of their time here.

Inference usually drives:

  • per-request token cost;
  • latency;
  • prompt design;
  • output control;
  • logging and monitoring.

Training

Training changes the model’s weights by running large optimisation loops over data.

That usually means:

  • much more data;
  • much more compute;
  • longer build times;
  • more specialised ML ops;
  • stronger data governance requirements.

Training is not what most teams mean when they casually say “make the model smarter”.

Fine-tuning

Fine-tuning starts from an existing model and adapts it to a narrower task, style or domain.

It can help when you have:

  • a stable task;
  • enough high-quality examples;
  • repeated prompts that are hard to solve with prompting alone;
  • a need for consistent tone or structure.

It is less useful when the problem is missing context, bad retrieval or unclear business rules.

A simple comparison

TermWhat changes?Typical goalCommon risk
InferenceNothing in the model weightsAnswer a live requestSlow, expensive or inconsistent prompts
TrainingThe model weightsBuild a new model or capabilityVery high cost, data burden, long timelines
Fine-tuningThe model weights, but on a narrower adaptation runBetter behaviour on a defined taskOverfitting, stale data, false confidence

The comparison sounds neat because the categories are neat. Real products are less neat. Many systems use inference plus retrieval, prompting, rules and review before they ever need fine-tuning.

Where teams get it wrong

Common mistakes include:

  1. Using fine-tuning to compensate for poor retrieval. Fine-tuning teaches the model tone and style; it does not teach it facts it never saw in training. A team fine-tuned a model on 30 customer support tickets hoping it would learn the correct answers to product-specific questions. The model learned the tone of support responses, but still hallucinated pricing and availability. The fix was retrieval-augmented generation, not more fine-tuning data.

  2. Using training language when they really mean prompt changes.

  3. Assuming fine-tuning will fix factual accuracy in the same way that adding data to a knowledge base would.

  4. Treating one small adapter run as if it were a full model redesign.

  5. Ignoring the data and governance work that fine-tuning still needs — data cleaning, labelling consistency, evaluation splits, drift monitoring.

  6. Assuming a model will keep the new behaviour forever without re-checking. Base-model updates can break fine-tuned behaviour silently [1][4].

If the base problem is missing context, a better prompt or better retrieval may beat fine-tuning on time, cost and maintenance.

Practical decision check

Ask these questions before you choose:

  • Is the task stable and repeated often enough to justify adaptation?
  • Can prompt design or retrieval solve it first?
  • Do we have enough high-quality examples? OpenAI’s fine-tuning docs recommend at least 50–100 high-quality examples before the results become reliable [1, §Preparing your dataset].
  • Is the target behaviour narrow and testable?
  • Can we measure whether the change actually helped?
  • Can we safely retrain or roll back if the behaviour drifts?

If you cannot answer those, the project is not ready for fine-tuning.

What this page cannot tell you

This page cannot tell you which provider or method will be cheapest for your case.

It cannot tell you:

  • how much data you need;
  • whether your examples are clean enough;
  • whether a fine-tune will improve the exact metric you care about;
  • whether a model provider changes its fine-tuning API or pricing;
  • whether your task is better solved by retrieval, rules or a human review step.

It can only help you avoid the category error.

What would change the advice

The guidance to prefer prompting or retrieval over fine-tuning assumes you have the data, the evaluation and the stability to make fine-tuning worthwhile. That assumption breaks down when:

  • The task changes frequently. Fine-tuning locks in behaviour from a snapshot of the data. If your task, domain or user base shifts regularly, the fine-tune will decay and need retraining — often faster than you can afford. A prompt or retrieval-based approach adapts by changing instructions or data, not model weights.

  • You have fewer than 50 high-quality examples. Fine-tuning on small, noisy or unrepresentative datasets often produces a model that sounds confident about the wrong things [1]. Below that threshold, prompting with a handful of examples (few-shot) or a structured system message will almost certainly outperform a fine-tune on cost and reliability.

  • A provider deprecates the base model your fine-tune depends on. Several providers retire old base models periodically. If your fine-tune was built on a model that is no longer available, you may lose the ability to run inference or retrain. Hugging Face’s model hub tracks deprecation dates for open-weight models [4]; commercial provider deprecation schedules are less transparent.

  • Prompting or retrieval improvements achieve the same quality gain for less maintenance. Before committing to a fine-tune, run an A/B test: can a better system message, 5–10 few-shot examples, or a retrieval step close the same gap? If yes, skip the fine-tune.

Regional caveats

The decision framework above is universal, but the practical options vary by jurisdiction:

  • UK/Europe: Data residency rules affect where training and fine-tuning data can be processed. Several providers (AWS Bedrock in Frankfurt, Google Vertex AI in London) offer fine-tuning within EU data boundaries. Check your provider’s data-processing addendum before uploading customer data.

  • US: Fine-tuning APIs are widely available but provider terms vary on whether your training data is used for model improvement. Check the data-usage opt-out settings for each provider.

  • Asia-Pacific: Provider fine-tuning availability is more limited. Some providers restrict fine-tuning to enterprise tiers or specific region endpoints. Check availability before committing to a workflow that depends on it.

Methodology and sources

Check date: 2026-05-22

What was checked: provider fine-tuning and training documentation, plus general ML glossary references.

What the sources were used for:

  • the difference between inference, training and fine-tuning [2][3];
  • the kinds of constraints and costs that belong to each [1][4];
  • the caution that model adaptation is not a substitute for prompt, retrieval or evaluation work [1].

Assumptions and limits:

  • provider terminology differs;
  • fine-tuning support changes over time;
  • this page is operational guidance, not a universal procurement rule;
  • the best answer depends on data quality, task stability and evaluation discipline.

Change log

  • 2026-05-25: revised per editorial review (LLM-0077). Integrated Editor’s Notes, added inline citations, fixed related-guide links to production routes, added “Where teams get it wrong” scenario (fine-tuning on 30 tickets for factual answers), added “What would change the advice” section, replaced Global applicability with regional caveats.
  • 2026-05-22: first draft built from the llm-editor-approved brief, with a clear term split, a decision table, and a practical check for when adaptation is actually justified.

Source list