Finetuning: When It Pays Off
What training actually changes, and the decision framework — because most teams reach for finetuning exactly when they shouldn't.
Finetuning continues training on your examples, nudging weights toward your distribution. Supervised finetuning (SFT) teaches from input→output demonstrations; preference methods like DPO teach from chosen-vs-rejected pairs. Understand what it changes: behavior — style, format, task-specific skill on a narrow distribution. What it does NOT do well: inject facts. Knowledge lives awkwardly in weights, goes stale instantly, and can't cite sources. Facts are RAG's job; behavior is finetuning's.
When it pays off: a consistent style or format you'd otherwise re-prompt with thousands of tokens every call (finetune it in, shrink the prompt); a narrow repeated task (your ticket taxonomy, your codebase's conventions) where a small finetuned model matches a frontier model at a fraction of the cost and latency; edge/local constraints where the model must be small; and distillation — training a small model on a big model's outputs for one task. The pattern: finetuning trades upfront work for per-call efficiency on a stable, high-volume task.
When it doesn't: fresh or per-user knowledge (RAG), tasks that shift monthly (retraining treadmill), tiny datasets (under a few hundred quality examples, prompting usually wins), and 'the prompt isn't working' (fix the prompt and evals first — cheaper iterations, same ceiling most of the time). The prerequisite everyone skips: an eval set BEFORE training. Without one you cannot know if the finetune helped, hurt, or quietly broke something the base model did fine.