The most expensive mistake in local AI is not a failed training run — it's a successful training run that solves the wrong problem. Before your dragon eats a single example, you need a precise training goal and proof that fine-tuning is even the right tool. This lesson gives you a decision framework used by teams who ship fine-tuned models for real.
The Three Tools, Honestly Compared
When a model doesn't do what you want, you have three escalating options:
1. Prompting (free, instant). Write better instructions, add examples in the prompt (few-shot), define the output format. This fixes more problems than most people believe.
2. RAG — Retrieval-Augmented Generation (cheap, fast to build). Store your documents in a search index; retrieve relevant chunks and paste them into the prompt at question time. This is the correct tool whenever the problem is the model doesn't know my facts.
3. Fine-tuning (this course). Actually change the model's weights with training examples. The correct tool when the problem is the model doesn't behave the way I need — wrong tone, wrong format, ignores instructions specific to your task, doesn't speak your domain's language.
Concept
The one-line rule the whole industry uses: fine-tuning teaches behavior, RAG provides knowledge. Facts change — put them in a database. Behavior is stable — bake it into the weights. Many production systems use both: a fine-tuned model plus RAG.Here is the decision as a flowchart:
Good and Bad Fine-Tuning Goals
Goals fine-tuning is great at:
- "Answer support questions in our brand voice, always ending with a next step." — tone + format ✔
- "Turn messy meeting notes into our exact status-report template." — structured transformation ✔
- "Classify incoming emails into our 12 internal categories." — task discipline ✔
- "Write product descriptions that sound like our catalog, in Romanian." — style + language ✔
- "Always output valid JSON matching our schema." — format reliability ✔
Goals that will end in tears:
- "Teach the model our 2026 price list." ✘ — facts change; use RAG. The model will hallucinate prices with total confidence.
- "Make a 4B model as smart as Claude." ✘ — fine-tuning doesn't add reasoning capacity; it shapes what's already there.
- "Train it on all our documents so it knows everything." ✘ — dumping raw documents into fine-tuning teaches the model to imitate your documents, not to answer questions about them.
Honest Note
If someone tells you they fine-tuned a model "to know their company data", either they actually built RAG, or their model confidently invents company data. Fine-tuning on raw documents is the single most common beginner mistake in local AI — and now it's one you'll never make.Our Course Project: Define Ember's Job
For the rest of this course, Ember's training goal is one of the strongest use cases for a local model:
Ember is a customer-support assistant for a small online shop. It answers in the shop's friendly-but-precise voice, in the customer's language, always structures answers as: short answer → explanation → concrete next step, and refuses to invent order details it doesn't have.
Notice what this goal is made of: voice (behavior), structure (format), discipline (refusing to hallucinate). All weight-worthy. The facts about specific orders would come from a database at runtime — not from training.
Write a Goal Card Before You Train Anything
Pro Tip
Professionals write the evaluation before the training. If you can't describe how you'll measure success, you don't have a training goal yet — you have a wish.Your goal card has four lines. Here's Ember's:
- Task: answer customer-support questions for an online shop
- Behavior: friendly-precise voice; structure: answer → explanation → next step; never invent order data
- Success test: on 30 held-out test questions, ≥25 follow the structure and 0 invent facts
- Non-goals: product knowledge (RAG's job later), languages beyond EN/RO