A dragon's size is limited by its wingspan; your model's size is limited by memory. This lesson gives you the exact memory math for training, then compares our two course machines — the Mac Studio M3 Ultra with 96 GB unified memory and the i9 PC with an RTX 4090 (24 GB VRAM) — so you know precisely which models each one can train, and which one wins at what.
The Memory Math That Decides Everything
Training memory is dominated by four things: the model's weights, the gradients, the optimizer state, and activations. How much you need depends entirely on the training method:
Concept
Rules of thumb per billion parameters:Full fine-tune (16-bit + AdamW): ~16–20 GB per 1B params — weights (2) + gradients (2) + optimizer (8–12) + activations.
LoRA (16-bit frozen base): ~2–2.5 GB per 1B params — the base just sits in memory; only tiny adapters train.
QLoRA (4-bit frozen base): ~0.6–0.8 GB per 1B params — the base is compressed to 4 bits, adapters still train in 16-bit.
Run the numbers and the picture is instant:
| Model size | Full fine-tune | LoRA (16-bit) | QLoRA (4-bit) |
|---|---|---|---|
| 1B | ~18 GB | ~3 GB | ~1.5 GB |
| 4B | ~70 GB | ~10 GB | ~4 GB |
| 8B | ~140 GB | ~18 GB | ~7 GB |
| 14B | ~250 GB | ~32 GB | ~12 GB |
| 32B | ~550 GB | ~70 GB | ~24 GB |
| 70B | ~1.2 TB | ~150 GB | ~48 GB |
Now overlay our two machines' memory ceilings:
Machine 1: Mac Studio M3 Ultra, 96 GB Unified Memory
Apple's trick is unified memory: CPU and GPU share one giant pool. There is no "copy to VRAM" — the GPU sees all 96 GB (macOS lets you push roughly 75% of it to the GPU by default, more if you tune it). Training runs on MLX, Apple's ML framework, using the mlx-lm package.
Strengths:
- Capacity is king. QLoRA on a 32B model is comfortable. LoRA on 14B works. Even a 70B QLoRA is possible — slow, but possible. On a 24 GB card these are simply out of reach.
- Silent, tiny power draw (~160 W under load), sits on a desk.
- The same machine then runs big models fast for inference — ~800 GB/s of memory bandwidth is excellent for token generation.
Weaknesses:
- Raw training compute. The M3 Ultra GPU is strong, but the CUDA ecosystem is a decade ahead in training optimizations. Expect training runs 2–4× slower than a 4090 for models that fit in both.
- Some cutting-edge training tricks (fancy kernels, some quantization schemes) land on CUDA first, MLX later or never.
Machine 2: i9 PC + RTX 4090, 24 GB VRAM
The 4090 is a monster: ~1000 GB/s memory bandwidth, dedicated tensor cores, and the full CUDA stack — Unsloth, Hugging Face transformers + peft + trl, Flash Attention, bitsandbytes. The i9 and system RAM matter mainly for data preprocessing; the GPU does the real work.
Strengths:
- Speed is king. For anything ≤ 8B, the 4090 rips through training. A LoRA run that takes an hour on the Mac finishes in ~15–20 minutes.
- Unsloth's kernels roughly double training speed and halve memory vs vanilla Hugging Face — a free 2× on this exact card.
- Every tutorial, every new technique, every optimization ships CUDA-first.
Weaknesses:
- 24 GB is a hard wall. 14B LoRA in 16-bit doesn't fit; 32B QLoRA barely squeezes with a small batch and short context. When it doesn't fit, it just crashes — there is no "slow but works" like on unified memory.
- Power and noise: 450 W on the GPU alone under load.
So Which One Should Train What?
| Scenario | Winner | Why |
|---|---|---|
| LoRA/QLoRA on 1B–8B (our course project) | RTX 4090 | 2–4× faster, everything fits anyway |
| QLoRA on 14B–32B | M3 Ultra | Fits with room to breathe; 4090 chokes |
| Anything touching 70B | M3 Ultra | Only machine with the memory, patience required |
| Serving the trained model afterwards | Tie | 4090 faster for small models, Mac handles bigger ones |
| Experimenting with brand-new techniques | RTX 4090 | CUDA-first ecosystem |