Fine-tuning basics (TypeScript) · Chapter 19 of 20
Capstone: a tagger Mangala trusts
By the end of this chapter you have trained Chulha's tagger from the given splits with everything this lab built (LoRA, oversampling, replay, early stopping that keeps the best epoch), saved the adapters with the story of how they were chosen, and proved on the held-out test set, once, that they clear Mangala's bar. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Proof
- 3 tasks
- TypeScript
- In the all labs plan
The first 2 chapters of Fine-tuning basics (TypeScript) are free. This one comes with the all labs plan.
Why you need this
Irfan's ready model has been tagging Chulha's messages in a trial. It is accurate and cheap, and it missed 4 of the 18 test leaks. Chapter 18 showed the best prompt catching every one of them. Chapter 17 showed the other cost: merged, the tagger wrecks the inbox's type-ahead.
Mangala writes down what she will accept before she sees any result:
| Mangala's bar | Why |
|---|---|
| accuracy above 90.9% | better than the best prompt, or why fine-tune at all |
| every leak caught (18/18) | a missed leak is a kitchen full of gas |
| format at least 98% | the router compares exact codes |
| type-ahead loss at most 1.30 | chota-base is 1.1614: its first job must survive |
Rashmi adds one line: "And I want to know how you got it, so we can do it again next quarter." Every piece you need was built in an earlier chapter. This one puts them in order and proves the result.
What you build: 3 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
oversample.ts: more leaks to learn from
Export oversample(set, times): a new list with each example of set, in order, repeated times[its tag] times (1 when the tag is not named, 0 leaves it out), its copies right after it.
Under if (import.meta.main), print for examples('train') and oversample(train, { LEAK: 2 }) one line per tag in TAGS order, then the total:
LEAK 94 -> 188
total 1000 -> N
- Task 2
tagger.ts: train the tagger, keep the best, save it
Export trainTagger(o) with o { train, val, epochs, patience, seed, times, replay, lr }, using the given pieces in lora.ts in this order: rows = oversample(o.train, o.times); run = startLora(o.seed); for each epoch from 1, train = trainEpoch(run, rows, o.replay, o.lr), val = valLoss(run, o.val), record { epoch, train, val }; a val lower than the best so far makes this the best epoch and snapshots copyAdapters(run.adapters); otherwise stop once o.patience epochs have passed since the best. Return { best, bestEpoch, epochs }.
Under if (import.meta.main), run Irfan's recipe on train and val (epochs 6, patience 2, seed 0, times { LEAK: 2 }, replay 8, lr 0.004), print each epoch and the best, and save C.saveAdapters(best, { base: 'chota-base', rank: 8, alpha: 16, bestEpoch, epochs }) to tagger-adapters.json. Run it once in the terminal (about half a minute): the check reads the file, and also runs trainTagger on small sets of its own.
- Task 3
report.ts: Mangala's bar
Load tagger-adapters.json (C.loadAdapters) and measure it on chota-base with the given evaluate and typeaheadLoss (lora.ts) on examples('test'), once. Print:
accuracy A%, format F%, LEAK C/L, type-ahead T (one decimal, and four for T)
Mangala's bar: met (or: Mangala's bar: not met (accuracy, LEAK, format, type-ahead), only those that failed, in that order)
The bar: accuracy above 90.9% (the best prompt), every leak caught, format at least 98%, type-ahead at most 1.30. The check measures your saved adapters against it too.
What you will learn
- A recipe from parts: which piece fixes which number, measured one piece at a time.
- Early stopping that keeps the best: a snapshot, not the last weights.
- Adapters with their story: the file says how it was chosen.
- One honest test: the test set touched once, on adapters already chosen.
More in Proof
Build "Capstone: a tagger Mangala trusts" yourself
Chulha, a Nagpur startup, tags WhatsApp messages for 40 LPG gas agencies, and a missed gas leak is not an option. You fine-tune chota, a tiny transformer, until it beats the best prompt while forgetting little. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of Fine-tuning basics (TypeScript) are free. This one comes with the all labs plan.