# Dragon Food: Building a Dataset That Actually Works

You can have perfect hardware and perfect hyperparameters, and a bad dataset will still give you a bad model — every single time. Data quality is the highest-leverage hour you will spend in this entire course. This lesson shows you the format, the quantity, and the cleaning rituals that separate a sharp fine-tune from an expensive disappointment.

## The Format: Chat-Style JSONL

Modern instruction fine-tuning uses one training example per line, in JSON, mimicking a real conversation. This is the format both MLX and Unsloth expect:

```json
{"messages": [
  {"role": "system", "content": "You are Ember, the support assistant for Nordlys Shop. Friendly and precise. Structure: short answer, explanation, next step. Never invent order details."},
  {"role": "user", "content": "My order hasn't arrived and it's been 10 days. Order #4571."},
  {"role": "assistant", "content": "I'm sorry about the wait — 10 days is longer than it should be.\n\nStandard delivery is 3–5 business days, so #4571 deserves a proper look. I can't see live courier data from here, so I won't guess where the package is.\n\nNext step: reply with your postal code and I'll open a trace request with the courier today. You'll get a tracking update within 24 hours."}
]}
```

Look closely — this one example teaches *four* behaviors at once: the apologetic-but-composed tone, the three-part structure, the refusal to invent tracking data, and the concrete next step. **Every example is a lesson; make each one teach something.**

<div class="concept-box">
<h4>Concept</h4>
The model learns to imitate the <strong>assistant</strong> turns, in the context of the system and user turns. Your dataset is literally a collection of "here is the perfect answer" demonstrations. If a demonstration is mediocre, you are training mediocrity — with great precision.
</div>

## How Many Examples Do You Need?

The honest answer surprises beginners in both directions:

| Goal type | Examples needed |
|---|---|
| Tone/voice adjustment | 50–200 |
| Format discipline (structure, JSON schema) | 100–500 |
| Task specialization (classification, extraction) | 300–2,000 |
| New domain style (legal, medical writing) | 1,000–10,000 |

For Ember, **150–300 excellent examples** beat 3,000 scraped mediocre ones. LoRA fine-tuning is sample-efficient; it's a scalpel, not a firehose.

<div class="honest-note">
<h4>Honest Note</h4>
"More data is always better" is pre-training wisdom, not fine-tuning wisdom. In fine-tuning, <strong>consistency beats volume</strong>. Ten examples that contradict your format teach the model that the format is optional. One hundred flawless examples teach it the format is law.
</div>

## Where Ember's Examples Come From

Three sources, in order of value:

**1. Real history (best).** Real customer emails + your best agent's real answers, cleaned up. Reality has distribution: weird questions, typos, anger, edge cases.

**2. Synthetic generation (good, with care).** Use a strong model (Claude, or a big local model) to draft examples: *"Generate 20 realistic customer questions about returns for an online shop, with answers in this exact voice and structure: [paste 3 real examples]"*. Then **hand-review every single one**. You're the head trainer; the big model is just your assistant.

**3. Templated edge cases (essential seasoning).** Hand-write the dangerous ones: the customer asking for order info you don't have (teach the refusal), the furious message (teach composure), the question in Romanian (teach language matching). These rarely occur in bulk data but define your model's character when it counts.

## The Cleaning Ritual

Before training, run every example through this checklist:

- **Would you ship this answer to a real customer?** If not, fix it or delete it.
- **Does it follow the exact target structure?** All of them. No exceptions — exceptions are anti-training.
- **Is the system prompt identical everywhere?** Same wording, byte for byte. You'll use the same one at inference time.
- **No secrets or personal data.** Real customer names, emails, order numbers get replaced with fakes. The weights *will* memorize them.
- **Deduplicate.** Near-identical examples overweight one lesson and bore the dragon.

## Split Before You Train: train / valid / test

Cut your data three ways **before** training, and never let the model see the last slice:

- **train.jsonl** (~80%) — what the model learns from
- **valid.jsonl** (~10%) — watched *during* training to catch overfitting (lesson 6)
- **test.jsonl** (~10%) — touched only at the very end, to grade the final dragon honestly

Both toolchains read this layout directly. MLX expects a folder with exactly these filenames; with Unsloth you'll load the same files with the `datasets` library.

<div class="pro-tip">
<h4>Pro Tip</h4>
Make the test slice <em>hard on purpose</em>. Put your nastiest edge cases there: the trap questions, the angry multi-part complaint, the request to invent data. A model that passes a hard test is trained; a model that passes an easy test is lucky.
</div>

<div class="try-it">
<h4>Try It</h4>
Build your v1 dataset this week: 10 hand-written gold examples (your absolute best answers), then 100–150 synthetic ones generated from those gold examples, then the cleaning ritual, then the three-way split. Target: <code>data/train.jsonl</code>, <code>data/valid.jsonl</code>, <code>data/test.jsonl</code>. Keep the counts in a note — you'll iterate on this in lesson 6.
</div>

<div class="checkpoint">
<h4>Checkpoint</h4>
You have chat-format JSONL data, split three ways, cleaned like you mean it. You know that 200 consistent examples beat 2,000 sloppy ones. Next lesson, the fun part: your dragon's first real training run.
</div>