Fine-tuning
Also: 微調 · fine-tuning · fine tuning · LoRA · 模型微調 · PEFT
Taking an existing model and continuing training on your own data, writing new behaviour or style into the weights.
When you will meet it
You arrive at this decision point when "the model keeps answering in the wrong format" and someone suggests "just fine-tune it". Not knowing what fine-tuning is for, you will spend weeks curating data and burning training budget on a problem a better prompt would fix — or worse, expect fine-tuning to teach the model internal knowledge it lacks (that is RAG's job), and end up with something expensive and stale.
An analogy
Three ways to shape an employee: hand them a playbook (prompt — instant, editable anytime); give them a reference library (RAG — knowledge stays current); or send them back for retraining (fine-tune — it rewires instinct and habit: deep, but slow, expensive, and obsolete the moment company policy changes).
Minimal example
決策順序（幾乎永遠從上往下試）：
1. 行為不對 → 改 prompt／系統提示 成本：幾分鐘
2. 缺你的文件知識 → 接 RAG 成本：幾天
3. 風格、格式、領域腔調要穩定且規模化，
提示詞已經壓不住 → 才考慮 fine-tune 成本：幾週起跳
隱藏成本：基座模型一換代，微調通常要重做；
每次上線前要有評測集，否則你不知道它把什麼改壞了。Fine-tuning is good at behaviour: fixed output formats, consistent tone, domain idioms. It is bad at injecting facts — anything learned is baked into the weights, and when it goes stale or wrong you cannot patch it the way you would patch a document.
What people get wrong
- Fine-tuning to inject knowledge, assuming training on company documents means the model "learned" them. It more likely absorbs the documents' tone while blurring the details — with no citations and no way to update. Knowledge goes to RAG, behaviour to fine-tuning; keep that line sharp.
- Treating fine-tuning as a one-off project. It is a pet, not a rock: new base-model generations force retraining, data needs ongoing curation, and every retrain must pass evals to prove nothing regressed. Many teams ship version one, lose the maintenance budget, and watch it rot.