Fine-tuning basics · Chapter 10 of 20

Inside chota-base

By the end of this chapter you can say what a language model is made of and count it, get its prediction for the next piece of text as probabilities, see exactly which predictions fine-tuning trains, and explain from the numbers why chota-base, before fine-tuning, answers a tagging request with nothing. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Inside
  • 4 tasks
  • Python
  • In the all labs plan

The first 2 chapters of Fine-tuning basics are free. This one comes with the all labs plan.

Why you need this

So far a fine-tuning job has been a box: files in, a model id out, some losses in between. Irfan wants to stop guessing about what the box does. The next chapters change chota's weights with your own code, and you cannot change something you have never looked at.

There is also Mangala's question from chapter 1, still unanswered. chota-base read two years of Chulha's customers' messages. Ask it to tag one and it says nothing at all. Why?

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    count.py: what the 45,152 numbers are

    Write param_groups(model): add up the sizes (w.size) of model.params by name: embeddings ("tok", "pos"), attention (names ending .wq, .wk, .wv, .wo), feed_forward (.fc_w, .fc_b, .proj_w, .proj_b), layer_norm (every other name with "ln" in it), and total (every weight). Return a dict {"embeddings", "attention", "feed_forward", "layer_norm", "total"}; work from the names, as the check also tries a model of another shape.

    Under if __name__ == "__main__":, print one line per group for chota-base, in that order:

    embeddings: N

  2. Task 2

    next_token.py: what chota expects next

    Write next_tokens(model, text, k): run forward(model, [[USER, *encode(text)]]), take the last position's scores (logits[0, -1], 721 of them), turn them into probabilities with softmax (exp(score - max) divided by the sum over the row), and return the k most likely, most likely first, as dicts {"piece": pieces([id])[0], "p": probability}.

    Under if __name__ == "__main__":, print the 5 most likely pieces after "delivery" for chota-base, one per line:

    PIECE P% (one decimal)

  3. Task 3

    template.py: where the loss is taken

    Write trained_pairs(messages): with row = chat_ids(messages), every position t where make_batch([row]).weights[0][t] is 1, as the pair (pieces(row.ids)[t], pieces(row.ids)[t + 1]).

    Under if __name__ == "__main__":, for user "cylinder kab aayega" and assistant "DELIVERY", print the pieces joined with spaces, then one line per pair:

    PIECE -> NEXT

  4. Task 4 · interview question

    Interview: why_silent.py, why does chota-base say nothing?

    Mangala: "Irfan says chota-base read two years of our customers' messages. Why does it answer nothing?" For each of the first 3 test messages, take prompt_ids(message_of(e)), the probabilities of the next piece at its last position, and print the most likely:

    MESSAGE: after <bot>, PIECE P% (one decimal)

    Then answer every test message with generate_many(model, messages, 4) and print:

    chota-base replied with nothing to N of M test messages

What you will learn

  • Weights: what chota's 45,152 numbers are, by kind.
  • The next piece: scores for every piece in the vocabulary (logits), turned into probabilities (softmax).
  • Greedy decoding: how a reply is made one piece at a time.
  • What is trained: the chat template's marks, and which predictions the loss counts.
  • The loss: how wrong a prediction is, as one number, from the probability of the right piece.

More in Inside

Build "Inside chota-base" yourself

Chulha, a Nagpur startup, tags WhatsApp messages for 40 LPG gas agencies, and a missed gas leak is not an option. You fine-tune chota, a tiny transformer, until it beats the best prompt while forgetting little. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Fine-tuning basics are free. This one comes with the all labs plan.