Dragon Food: Building a Dataset That Actually Works

How to Train Your Model Course by TEN INVENT · July 2026

You can have perfect hardware and perfect hyperparameters, and a bad dataset will still give you a bad model — every single time. Data quality is the highest-leverage hour you will spend in this entire course. This lesson shows you the format, the quantity, and the cleaning rituals that separate a sharp fine-tune from an expensive disappointment.

The Format: Chat-Style JSONL

Modern instruction fine-tuning uses one training example per line, in JSON, mimicking a real conversation. This is the format both MLX and Unsloth expect:

{"messages": [
  {"role": "system", "content": "You are Ember, the support assistant for Nordlys Shop. Friendly and precise. Structure: short answer, explanation, next step. Never invent order details."},
  {"role": "user", "content": "My order hasn't arrived and it's been 10 days. Order #4571."},
  {"role": "assistant", "content": "I'm sorry about the wait — 10 days is longer than it should be.\n\nStandard delivery is 3–5 business days, so #4571 deserves a proper look. I can't see live courier data from here, so I won't guess where the package is.\n\nNext step: reply with your postal code and I'll open a trace request with the courier today. You'll get a tracking update within 24 hours."}
]}

Look closely — this one example teaches four behaviors at once: the apologetic-but-composed tone, the three-part structure, the refusal to invent tracking data, and the concrete next step. Every example is a lesson; make each one teach something.

How Many Examples Do You Need?

The honest answer surprises beginners in both directions:

For Ember, 150–300 excellent examples beat 3,000 scraped mediocre ones. LoRA fine-tuning is sample-efficient; it's a scalpel, not a firehose.

Where Ember's Examples Come From

Three sources, in order of value:

1. Real history (best). Real customer emails + your best agent's real answers, cleaned up. Reality has distribution: weird questions, typos, anger, edge cases.

2. Synthetic generation (good, with care). Use a strong model (Claude, or a big local model) to draft examples: "Generate 20 realistic customer questions about returns for an online shop, with answers in this exact voice and structure: [paste 3 real examples]". Then hand-review every single one. You're the head trainer; the big model is just your assistant.

3. Templated edge cases (essential seasoning). Hand-write the dangerous ones: the customer asking for order info you don't have (teach the refusal), the furious message (teach composure), the question in Romanian (teach language matching). These rarely occur in bulk data but define your model's character when it counts.

The Cleaning Ritual

Before training, run every example through this checklist:

• Would you ship this answer to a real customer? If not, fix it or delete it.
• Does it follow the exact target structure? All of them. No exceptions — exceptions are anti-training.
• Is the system prompt identical everywhere? Same wording, byte for byte. You'll use the same one at inference time.
• No secrets or personal data. Real customer names, emails, order numbers get replaced with fakes. The weights will memorize them.
• Deduplicate. Near-identical examples overweight one lesson and bore the dragon.

Split Before You Train: train / valid / test

Cut your data three ways before training, and never let the model see the last slice:

• train.jsonl (~80%) — what the model learns from
• valid.jsonl (~10%) — watched during training to catch overfitting (lesson 6)
• test.jsonl (~10%) — touched only at the very end, to grade the final dragon honestly

Both toolchains read this layout directly. MLX expects a folder with exactly these filenames; with Unsloth you'll load the same files with the datasets library.