Fine-tuning basics (TypeScript) · Chapter 8 of 20

A fine-tuning job

By the end of this chapter you can run a fine-tuning job the way you would on a real provider: upload the files, start the job, follow it to the end without hammering the API, read what its events say, and use the model it makes. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: A fine-tuning job
  • 4 tasks
  • TypeScript
  • In the all labs plan

The first 2 chapters of Fine-tuning basics (TypeScript) are free. This one comes with the all labs plan.

Why you need this

Irfan's tags-v1 is a model he trained last week, and on the validation set it still misses 8 of 24 leaks. Mangala wants better, and Irfan has an idea: train longer. To try it, you need to run a job yourself.

A fine-tuning job is not a function call. You send files to a provider, ask it to train, and wait: seconds for chota, minutes or hours for a big model. Meanwhile your code must know what the job is doing, show its progress, notice when it fails, and pick up the new model when it is done. Nothing you have written so far does any of that.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    upload.ts: upload a training file

    Write and export upload(path): upload the JSONL file at path with client.files.create({ file: createReadStream(path), purpose: 'fine-tune' }) and return the new file's id.

    Under if (import.meta.main), upload train.jsonl, then val.jsonl, from Chulha's data folder (DATA in ../../shared/data.ts) and print one line for each:

    train.jsonl: FILE_ID

    val.jsonl: FILE_ID

  2. Task 2

    start-job.ts: start a fine-tuning job

    Write and export startJob(trainingFile, validationFile): create a job with client.fineTuning.jobs.create of model 'chota-base' on the two file ids, with suffix 'tags-v2' and method { type: 'supervised', supervised: { hyperparameters: { n_epochs: 4 } } }, and return the job's id.

    Under if (import.meta.main), upload train.jsonl and val.jsonl with your upload, start the job, retrieve it and print:

    started JOB_ID: STATUS

  3. Task 3

    follow.ts: follow a job until it finishes

    Write and export follow(jobId): until the job has finished, retrieve it, print the message of every event not printed yet, oldest first and each exactly once, and wait 500 ms; return the job once its status is succeeded, failed or cancelled. Events come newest first, at most 100 a page: when page.has_more is true, ask again with after set to the last event's id.

    Under if (import.meta.main), follow process.argv[2] (or the newest job when none is given) and then print:

    status: STATUS, model: FINE_TUNED_MODEL

  4. Task 4 · interview question

    Interview: use-it.ts, is tags-v2 better, and what did it cost?

    Rashmi: "You trained four epochs where Irfan trained three. Is it better than tags-v1, and what did the training cost?" In one script: upload train.jsonl and val.jsonl, start the job with your startJob, follow it to the end, then score 'ft:chota-base:chulha:tags-v1:c0ffee01' and your job's fine_tuned_model on examples('test') with scoreModel (./score.ts, given). After follow's lines, print:

    tags-v1: accuracy A%, format F%, LEAK C/L

    tags-v2: accuracy A%, format F%, LEAK C/L

    tags-v2 trained T tokens, ₹R

    (T is the job's trained_tokens; R = T × chota-base's training price per million / 1,000,000, two decimals)

What you will learn

  • Files: why a job trains on uploaded files, and the fine-tune purpose.
  • A job: the base model, the files, the suffix, and the hyperparameters (epochs, batch size, learning rate).
  • States: validating_files, queued, running, and how a job ends.
  • Polling and events: asking again and again, politely, and reading a list that comes newest first, a page at a time.
  • The model id: what ft:chota-base:chulha:... means and how to use it.

More in A fine-tuning job

Build "A fine-tuning job" yourself

Chulha, a Nagpur startup, tags WhatsApp messages for 40 LPG gas agencies, and a missed gas leak is not an option. You fine-tune chota, a tiny transformer, until it beats the best prompt while forgetting little. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Fine-tuning basics (TypeScript) are free. This one comes with the all labs plan.