Fine-tuning basics (TypeScript) · Chapter 3 of 20
Cleaning the export
By the end of this chapter you can look at a raw export before you train on it, turn every way a tag was typed into one code, clean messages into the form the model will read (with personal data masked), and account for every row you drop. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: The data
- 4 tasks
- TypeScript
- In the all labs plan
The first 2 chapters of Fine-tuning basics (TypeScript) are free. This one comes with the all labs plan.
Why you need this
Chapter 2 ended with a plain fact: the fine-tuned model is cheap and good, and it still misses leaks. Irfan's first question is not about the model. It is about the data: "a fine-tuned model learns exactly what its examples show it, mistakes included."
So far you used train.jsonl, which Irfan prepared. Now Mangala's desk sends the raw export of September and October, 2,600 rows, the way agents actually typed them, and you make the training data yourself. Chapters 3 to 7 are that pipeline: clean, remove duplicates, split without leaks, write the chat format, count tokens and cost. At the end your files will be byte for byte the ones the lab gave you, so you will know every decision inside them.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
tag-spellings.ts: how did the desk type the tags?
Read export.csv with readCsv (../../shared/data.ts) and count the rows of each tag exactly as typed (' REFILL' and 'Refill' are different tags here). Print, in this order:
N rows
"TAG" COUNT (one line per tag, the tag written with JSON.stringify, most rows first; the same count keeps the order of first appearance)
K spellings (how many different tags)
- Task 2
normal-tag.ts: one code for every way of typing it
Export normalTag(tag): trim the tag, turn runs of spaces into one, uppercase it. If that is one of the six TAGS, return it; if it is in Mangala's fixes table (REFIL, BOOKING to REFILL; LATE DELIVERY, DELIVRY to DELIVERY; LEAKAGE, GAS LEAK to LEAK; NEW CONNECTION, KYC, CONECTION to CONNECTION; PAYMNT, SUBSIDY to PAYMENT; OTHERS, MISC to OTHER), return the code it maps to; anything else ('', '?', 'spam') returns null.
Trap: ' late delivery ' has spaces at both ends and two inside.
- Task 3
clean.ts: clean a message the way chota reads it
Export clean(message), in this order: NFKC-normalize, lowercase and trim; return '' for '<media omitted>' or 'this message was deleted'; replace every match of /(\+91[\s-]?)?[6-9]\d{9}/g with [phone]; turn every character other than a to z, 0 to 9, the space and . , ? ! ' - / : ( ) [ ] & + @ # % into a space; turn runs of spaces into one and trim. clean('Mera number 9876543210 hai 🙏') is 'mera number [phone] hai'.
Trap: a 10-digit number that starts with 5 is not a mobile number and stays.
- Task 4 · interview question
Interview: drop-report.ts, what did cleaning throw away?
Irfan: "Tell me what you dropped and why." Go through the export with your clean and normalTag: a row is "no text" when clean(message) is '', otherwise "no tag" when normalTag(tag) is null, otherwise kept. Print five lines:
raw N
no text N
no tag N
kept N
phone numbers masked in N (kept rows whose cleaned text has [phone] in it)
What you will learn
- Look first: count what is really in a column before you change it.
- Normalise labels: one code for every way of typing it, and null when there is none.
- Clean text: the same message always in the same form, with phone numbers masked.
- A drop report: what you threw away, and why, as numbers.
More in The data
Build "Cleaning the export" yourself
Chulha, a Nagpur startup, tags WhatsApp messages for 40 LPG gas agencies, and a missed gas leak is not an option. You fine-tune chota, a tiny transformer, until it beats the best prompt while forgetting little. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of Fine-tuning basics (TypeScript) are free. This one comes with the all labs plan.