Local or nothing syllabus
The honest version of "just run it locally" — what fits, what it costs, and the point where it stops being worth it.
Module 0 — What actually fits
Before anything else: whether your hardware is in the game at all. Free, and it should be.
- Reading a model card for what it doesn't say — Parameter counts are marketing. Memory at your context length is the constraint. (1 min)
- VRAM arithmetic you can do in your head — Weights, KV cache, overhead — and which one surprises you. (2 min)
Module 1 — Quantisation without the folklore
Everyone repeats that 4-bit is fine. Measure it on your task instead.
- What quantisation costs you, measured — Not perplexity. Your harness, your probes, one run per format. (1 min, paid)
- Picking a format for your hardware — The right answer differs on Apple silicon, on a consumer GPU, and on a server card. (1 min, paid)
Module 2 — The cost model
Where local stops being cheaper, stated as a break-even you can defend.
- Tokens, watts and wall-clock — A machine that runs all day has an electricity bill and a depreciation schedule. (1 min, paid)