insideout Get Pro

09 · Prompt, retrieve or fine-tune

Pro

Training and Tuning, lesson 9

Prompt, retrieve or fine-tune

How to choose between the three ways to adapt a model to your task.

You know how fine-tuning changes a model ("Instruction tuning" and "Adapters and LoRA"). It helps to know roughly what retrieval does: look up relevant text and put it in the prompt. This lesson is about when to reach for which.

Part 1. Three ways to change what a model does

Your support assistant has three complaints logged against it. It quoted last month's refund policy. It writes long friendly paragraphs when the ticket system needs a strict JSON reply. And at your volume, the large model is costing more than the support team it was meant to help.

You have three tools. Prompting: change the instructions and examples you send. Retrieval: look up the right documents for each question and put them in the prompt. Fine-tuning: train the model further on your own examples, so the change lives in its weights.

Your turn

Which tool fits the first complaint, the outdated refund policy?

The model is missing a fact that changes over time. Retrieval puts today's version in front of it on every request, and when the policy changes again you update a document, not a model. Fine-tuning would bake in this month's policy and go stale the same way.

Part 2. Fine-tuning changes behaviour, retrieval supplies knowledge

Keep going with Pro

You've done the first part. Pro opens the rest of this lesson, and every other one.

Still to come
  1. Fine-tuning changes behaviour, retrieval supplies knowledge
  2. A decision order
  3. When fine-tuning pays for itself
See Pro plansI have Pro: sign in
Next lessonInside the RL step: policy gradients and PPOHow a reward becomes a change in the model's weights. Start lesson 10

Training and Tuning overviewSpotted a mistake? Tell us.