The- most- common- mistake- is- using- fine-tuning- to- compensate- for- poor- retrieval.- Fine-tuning- teaches- the- model- tone- and- style;- it- does- not- teach- it- facts- it- never- saw- in- training.- If- your- model- is- giving- wrong- answers- because- the- right- information- is- not- in- the- prompt,- fine-tuning- will- not- fix- it- —- it- will- just- make- the- wrong- answers- sound- more- confident.

##- A- simple- comparison

The- comparison- sounds- neat- because- the- categories- are- neat.- Real- products- are- less- neat.- Many- systems- use- inference- plus- retrieval,- prompting,- rules- and- review- before- they- ever- need- fine-tuning.

##- Where- teams- get- it- wrong

Common- mistakes- include:

1.- Using- fine-tuning- to- compensate- for- poor- retrieval.- Fine-tuning- teaches- the- model- tone- and- style;- it- does- not- teach- it- facts- it- never- saw- in- training.- A- team- fine-tuned- a- model- on- 30- customer- support- tickets- hoping- it- would- learn- the- correct- answers- to- product-specific- questions.- The- model- learned- the- tone- of- support- responses,- but- still- hallucinated- pricing- and- availability.- The- fix- was- retrieval-augmented- generation,- not- more- fine-tuning- data. 2.- Using- training- language- when- they- really- mean- prompt- changes. 3.- Assuming- fine-tuning- will- fix- factual- accuracy- in- the- same- way- that- adding- data- to- a- knowledge- base- would. 4.- Treating- one- small- adapter- run- as- if- it- were- a- full- model- redesign. 5.- Ignoring- the- data- and- governance- work- that- fine-tuning- still- needs- —- data- cleaning,- labelling- consistency,- evaluation- splits,- drift- monitoring. 6.- Assuming- a- model- will- keep- its- new- behaviour- forever- without- re-introspecting.- Base-model- updates- can- break- fine-tuned- behaviour- silently- [1][4].

If- the- base- problem- is- missing- context,- a- better- prompt- or- better- retrieval- may- beat- fine-tuning- on- time,- cost- and- maintenance.

##- Practical- decision- check

Ask- these- questions- before- you- choose:

Is- the- task- stable- and- repeated- often- enough- to- justify- adaptation? Can- prompt- design- or- retrieval- solve- it- first? Do- we- have- enough- high-quality- examples?- OpenAI’s- fine-tuning- docs- recommend- at- least- 50–100- high-quality- examples- before- the- results- become- reliable- [1]. Is- the- target- behaviour- narrow- and- testable? Can- we- measure- whether- the- change- actually- helped? Can- we- safely- retrain- or- roll- back- if- the- behaviour- drifts?

If- you- cannot- answer- those,- the- project- is- not- ready- for- fine-tuning.

##- What- would- change- the- advice

The- guidance- to- prefer- prompting- or- retrieval- over- fine-tuning- assumes- you- have- the- data,- the- evaluation- and- the- stability- to- make- fine-tuning- worthwhile.- That- assumption- breaks- down- when:

The- task- changes- frequently.- Fine-tuning- locks- in- behaviour- from- a- snapshot- of- the- data.- If- your- task,- domain- or- user- base- shifts- regularly,- the- fine-tune- will- decay- and- need- retraining- —- often- faster- than- you- can- afford.- A- prompt- or- retrieval-based- approach- adapts- by- changing- instructions- or- data,- not- model- weights. You- have- fewer- than- 50- high-quality- examples.- Fine-tuning- on- small,- noisy- or- unrepresentative- datasets- often- produces- a- model- that- sounds- confident- about- the- wrong- things- [1].- Below- that- threshold,- prompting- with- a- handful- of- examples- (few-shot)- or- a- structured- system- message- will- almost- certainly- outperform- a- fine-tune- on- cost- and- reliability. A- provider- deprecates- the- base- model- your- fine-tune- depends- on.- Several- providers- retire- old- base- models- periodically.- If- your- fine-tune- was- built- on- a- model- that- is- no- longer- available,- you- may- lose- the- ability- to- run- inference- or- retrain.- Hugging- Face’s- model- hub- tracks- deprecation- dates- for- open-weight- models- [4];- commercial- provider- deprecation- schedules- are- less- transparent. Prompting- or- retrieval- improvements- achieve- the- same- quality- gain- for- less- maintenance.- Before- committing- to- a- fine-tune,- run- an- A/B- test:- can- a- better- system- message,- 5–10- few-shot- examples,- or- a- retrieval- step- close- the- same- gap?- If- yes,- skip- the- fine-tune.