Ember is trained, tested, and quantized. The final lesson is about letting it fly in the real world: serving it as an app your tools can talk to, keeping it healthy over time, and — if you choose — sharing it with the community. This is where a training project becomes a working system.
Option 1: Ollama — Ember as a Local Service
Ollama turns a GGUF file into a locally served model with an OpenAI-compatible API. Create a Modelfile next to your GGUF:
FROM ./ember-q4_k_m.gguf
SYSTEM """You are Ember, the support assistant for Nordlys Shop. Friendly and precise. Structure: short answer, explanation, next step. Never invent order details."""
PARAMETER temperature 0.4
Note the system prompt is the exact one from your training data — this is why lesson 4 insisted on byte-identical system prompts. Then:
ollama create ember -v ./Modelfile
ollama run ember
Ollama also exposes http://localhost:11434/v1/chat/completions — an OpenAI-compatible endpoint. Any tool, script, or app that speaks the OpenAI API can now use Ember by changing one URL. Your model, your machine, zero per-token cost.
Option 2: LM Studio — Ember with a Cockpit
Drop the GGUF into LM Studio's models folder and you get a friendly chat UI, sampler controls, and its own local API server — the comfortable choice if you prefer buttons to terminals, and the easiest way to demo Ember to a non-technical colleague.
Pro Tip
Serving settings matter as much as training. Keep temperature low (0.3–0.5) for a support assistant — you trained discipline into the weights; don't let a hot sampler shake it back out. And set a sensible context limit: Ember answers tickets, it doesn't need 128k tokens of history.Wiring Ember into a Real Workflow
With an OpenAI-compatible endpoint, integration is one HTTP call:
import requests
def ask_ember(question: str) -> str:
r = requests.post("http://localhost:11434/v1/chat/completions", json={
"model": "ember",
"messages": [{"role": "user", "content": question}],
})
return r.json()["choices"][0]["message"]["content"]
From here, the patterns you already know from web development take over: a helpdesk form that drafts replies with Ember, a Slack bot, a batch job that pre-classifies overnight tickets. And remember lesson 2's promise — this is where RAG joins the party: retrieve the customer's real order data from your database, paste it into the prompt, and let your fine-tuned voice-and-discipline model present it. Fine-tuned behavior + retrieved facts is the production-grade combo.
Sharing on Hugging Face (Optional, Classy)
If your dataset contains nothing private, publishing your model back to the community is how the open ecosystem you just benefited from keeps existing:
pip install huggingface_hub
hf auth login
hf upload your-username/ember-4b-support ./ember-gguf
Publish the GGUF (most useful to others), or just the LoRA adapters (tiny — a few hundred MB), or both. Write an honest model card: base model, dataset size and nature, what it's good at, what it's not. The best model cards read like your goal card from lesson 2 — because they are one.
Honest Note
Before publishing, re-read lesson 4's privacy checklist with fresh eyes. Fine-tuned weights can leak training data — researchers extract verbatim examples from published models routinely. If your data came from real customers, publish the recipe (base + hyperparameters + synthetic examples), not the weights. When in doubt, keep the dragon home.Keeping the Dragon Healthy
A deployed model is not done — it drifts out of sync with reality as your products, policies, and customers change. The maintenance loop is just this course in miniature:
- Collect the real questions Ember answered badly (your users will happily tell you)
- Add corrected versions to the dataset — 20–50 new examples at a time
- Retrain the same pipeline (it's one command now), bump the version: ember-v4, v5…
- Re-run the same test set — your regression suite — before swapping the served model
- Every 6–12 months, consider rebasing onto the newest small model — base models improve fast, and your dataset carries all your work forward
Concept
Notice what your real asset is: the dataset, not the weights. Weights are compiled artifacts; the dataset is source code. New base model? Recompile. Better technique? Recompile. Guard the dataset in version control like the crown jewels it is.Try It
Ship the full loop end-to-end: serve your quantized Ember with Ollama, call it from a 10-line Python script, and answer one real question from your actual use case. Then take a moment — a model that you trained, on your machine, just answered in your voice over an API. That's the whole promise of this course, delivered.Where to Fly Next
- Scale the base: rerun your pipeline on an 8B–14B model (lesson 7's math says your machine can)
- Add RAG properly: vector database + retrieval in front of Ember's endpoint
- Try function calling: fine-tune structured tool-call outputs — same pipeline, new dataset
- Automate the loop: nightly retrain + eval, so the dragon feeds itself