Build your own coding agent · Chapter 16 of 20

A transcript you can replay

By the end of this chapter you can record every event of a Kaarigar run to a file as it happens, replay that run later with no model at all, and put two runs side by side to find the exact request where they went different ways. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Scale and trust
  • 3 tasks
  • Python
  • In the all labs plan

The first 2 chapters of Build your own coding agent are free. This one comes with the all labs plan.

Why you need this

On Tuesday Kaarigar fixed MD-108. On Wednesday, with one tool changed, it did not: it skipped two tests instead. Rukhsana wants to know where the two runs parted. Jaya wants something else: a record of what the agent did on her machine that outlives the program, every file it read and every result it was shown, for the day someone asks. And Kedar wants to test Kaarigar itself, the way MeterDown tests fares: run the same thing again and get the same answer, without paying for a model every time.

Kaarigar has run.messages, the whole conversation, but it lives in memory and is gone when the program ends. And it has no way to run again without the model: every new run is new requests, new rupees, and with a real model, often a different run. Real models are not deterministic, and a provider updates them; the run you want to look at again may never happen twice.

Coding agents keep transcripts for these reasons. Claude Code writes each session to a JSON Lines file you can resume; Codex keeps a rollout file per session. This chapter gives Kaarigar the same, and then uses it.

What you build: 3 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    recorder.py: record a run

    Write recorder(file): empty the file, and return a function for the agent's log hook that appends each event as one JSON line, with "at" (datetime.now(timezone.utc).isoformat()) first. Write events(file): every non-empty line, parsed. Under if __name__ == "__main__":, run run_agent on a checkout of MD-101 with issue_text("MD-101") and Agent(SYSTEM, tools(), log=recorder("/tmp/kaarigar-MD-101.jsonl")), then print:

    events: N

    start 1, request R, reply R, tool T, end 1 (each type once, in the order it first appears, with its count)

  2. Task 2

    replay.py: replay a run with no model

    Write a class ReplayClient(file): its create(**body) gives back the transcript's reply events in order, each as shaped(message, usage) (given), raises RuntimeError when the run asks for more replies than the file has, and its used() says how many it gave. Under if __name__ == "__main__":, record a run of MD-108 to /tmp/kaarigar-MD-108.jsonl, replay it on a fresh checkout of MD-108 with Agent(SYSTEM, tools(), client=ReplayClient(file)), and print:

    recorded: N requests (the first run's steps)

    replayed: N replies (the client's used())

    same tool calls: yes (or no)

    MD-108: fixed (or not fixed, the replay's checkout)

  3. Task 3 · interview question

    Interview: where_they_part.py, where do two runs part?

    Write did(message): a reply's tool calls as NAME(ARGUMENTS) joined with ", ", or answer: TEXT. Write first_difference(file_a, file_b): the first reply (counting from 1) where did() differs, as Difference(step, a, b), with "(ended)" for a side that has no reply there, or None. Under if __name__ == "__main__":, record MD-108 twice in fresh checkouts with SYSTEM: "report" with tools(), and "names only" with run_tests swapped for the given names_only tool. After each run print LABEL: MD-108 fixed, N requests (or not fixed). Then print:

    they part at request STEP

    report: A

    names only: B

What you will learn

  • Events, not just messages: what happened, in order, with times.
  • JSON Lines: one event per line, appended as the run goes.
  • A replay client that answers the loop from the file: zero requests to the model.
  • What a replay can and cannot tell you: it repeats the model, not your code.
  • Where two runs part: the first reply that differs, and why.

More in Scale and trust

Build "A transcript you can replay" yourself

MeterDown, a Pune app for auto-rickshaw fares, has eleven open issues. You build Kaarigar, an agent that fixes them without breaking tests, running risky commands or touching files outside the repository. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Build your own coding agent are free. This one comes with the all labs plan.