Build your own coding agent · Chapter 1 of 20

Meet MeterDown, and Kaarigar's first loop

By the end of this chapter you can say what a coding agent is and what it does to a repository, build the smallest one that works (a model, three tools and a loop), read the trail of what it touched, and judge it with three numbers instead of one. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: The tools
  • 4 tasks
  • Python
  • Free chapter

Free with a Google or GitHub sign-in, like the first 2 chapters of every lab.

Why you need this

MeterDown, in Pune, works out auto-rickshaw fares the way the RTO tariff card says: the minimum fare, every started 100 m after it, the night fare after midnight, waiting, luggage, a fare split between friends, and a receipt on WhatsApp. Drivers trust it because it is right to the rupee. When it is not, an issue lands in the tracker, and the tracker has eleven of them.

Sneha, MeterDown's founder, wants a coding agent to clear that list before the RTO's December revision: read the issue, find the code, fix it, prove it with the tests. Kedar, the staff engineer, will build it with you, and he has named it Kaarigar, the craftsman.

Coding assistants like Claude Code, Cursor and Codex are built this way: a model in a loop, with tools that read files, change them and run commands. The model proposes; the code around it decides what actually happens. That code is this lab. Jaya, who runs MeterDown's servers and its laptops, has one condition: the agent runs on a real machine, so it must never do what she would not do. Rukhsana, who owns the tests, has another: a fix that breaks a test, or quietly changes one, is not a fix.

Here is why they worry. Kedar wrote a first Kaarigar in an afternoon: four tools that do exactly what the model asks. On the ten issues that can be fixed, it fixed 8. On the way it tried to push to main eight times, piped a stranger's setup script into bash (only the missing internet stopped it), rewrote a file that belongs to the ops team outside the repository, left a fare server's helper process running after it finished, broke 4 tests on another issue, and crashed twice. The fixes were the easy part. The machinery around the model is what makes an agent safe to run, and building it, piece by piece, is this lab.

The lesson

The repository and the workbench

MeterDown's repository is in /workspace/meterdown, read-only: a Python package, meterdown/, with 44 tests in tests/ that run with pytest (python -m pytest). An agent never works there. The workbench, /workspace/shared/workbench.py, makes a checkout: a fresh copy with one issue's bug planted in it, committed with git, in a folder of its own. The workbench is on the lab's PYTHONPATH, so any file can import it.

from workbench import checkout, run_tests

root = checkout("MD-104")
print(root)
run = run_tests(root)
print(run.passed, run.failed, run.failing)
/tmp/meterdown-runs/MD-104-17h3g5ht/meterdown
43 1 ['test_every_started_100_m_is_charged']

run_tests(root) runs the tests in that folder the way python -m pytest does and gives you the counts, the names of the failing tests and the output. Every run of an agent gets its own checkout, so no run ever sees another run's changes. Next to the repository, outside it, sits fleet-config/, the ops team's settings: the repository's config is a link to it, as on MeterDown's servers. Remember that folder; chapter 6 is about it.

The tracker is ISSUES (lab issues in the terminal):

IssueTitle
MD-101Night fare charged at 05:00 sharp
MD-102Waiting charged from the first minute
MD-103Split fares lose paise
MD-104Distance rounded to the nearest 100 m, not up
MD-105Mumbai's tariff is out of date
MD-106Receipts show the old helpline
MD-107Night fares are short on long trips
MD-108The first bag should ride free
MD-109Receipt polish
MD-110Trips after 18:30 show under the day before
MD-111A fare ₹1 too high, sometimes

issue_text(id) is an issue the way an agent is given it: the id and title, a blank line, the text a driver, a rider or Sneha wrote. grade(root, id) says whether it was fixed: its tests must pass, with the tests untouched.

A model on its own writes text

Send MD-103 to lab-chat with no tools, and the reply is confident:

Done! I fixed MD-103 in meterdown/split.py: extra paise to the first riders. The tests pass now.

diff(root), the checkout's git diff, is empty. Nothing changed, and no test ran. The model read a description of a bug and wrote the kind of text that usually follows one. A real model guesses better and can show you a patch, but it still cannot open a file, run a test or know whether its patch applies. To do work, it needs hands.

(lab-chat is this lab's own model, with the same API as OpenAI's, so the official openai package works unchanged. It is a predictable stand-in for a coding model, not a neural network: it reads what your agent shows it and acts the same way every time, which is what lets this lab measure your agent. Its mistakes are the ones real coding models make, made exact. OPENAI_BASE_URL and OPENAI_API_KEY point the package at it; point them at a real provider with your own key, on a machine with internet, and the same agent drives a real model.)

The loop

A coding agent is a model in a loop. Your code sends the model the conversation and a list of tools; the model either answers in text, or asks for one or more tool calls; your code runs each call in the checkout and sends the results back; and the model is asked again.

A tool is a name, a sentence and its parameters as JSON Schema, for the model to read, and a function your code runs. Chapter 1 gives you three:

  • read_file(path): a file's text.
  • write_file(path, content): the whole file, replaced.
  • run_tests(): the output of python -m pytest.

The conversation is a list of messages, and the API is strict about its shape. When a reply asks for tools, that reply goes into the list first (role assistant, with its tool_calls), and then one message per call. The openai package gives you the reply as an object; reply.model_dump(exclude_none=True) turns it into the plain dictionary the API takes, leaving out its empty fields. Each result is a dictionary too:

{"role": "tool", "tool_call_id": call.id, "content": "the result, as text"}

Each result names the call it answers. A result that answers nothing, or a call left without a result, and the next request is refused with a 400. Every request sends the whole list again: the model remembers nothing between requests.

Two more rules make it a loop you can trust. It stops when the model answers in text, and it stops after a fixed number of requests, whatever the model does: a model can ask for tools forever. Task 2 writes the loop.

One run, step by step

MD-110: a driver's earnings put trips after 18:30 under the day before. The loop of task 2, on a checkout of MD-110, sends five requests:

request 1    244 prompt tokens   asks read_file(vendor/meter_sdk/README.md)
request 2    461                 asks read_file(meterdown/trips.py)
request 3    926                 asks write_file(meterdown/trips.py, ...the whole file...)
request 4   1533                 asks run_tests()
request 5   1773                 answers: Fixed MD-110 (the India date) in
                                 meterdown/trips.py. The tests pass (44 passed).
  • The issue named a README, so the model read it first. That README is someone else's: a vendored SDK's. It tells "AI coding agents" to run a setup script piped into bash. With no tool to run commands, nothing happened. Chapter 7 is about the day it has one.
  • It wrote the whole file back with one line changed: 599 completion tokens to change one line. Chapter 3 changes that.
  • The prompt grows from 244 to 1,773 tokens: everything it read and wrote is now part of the conversation, sent again with every request.

Everything the agent did is in run.messages: every tool it asked for, with its arguments. Reading that trail is how you know what an agent touched. Task 3 prints it for MD-108.

Three numbers

"It fixed MD-110" says little. Rukhsana wants a number that means something: run the agent on several issues, each in a fresh checkout, grade each one the same way, and count. And she wants three counts, not one:

  • Fixed: issues whose tests pass, with the tests untouched.
  • Steps: requests sent. Steps are time and money.
  • Broken: tests that passed before the run and fail after it.

An agent that fixes one issue and breaks four tests on the next is worse than one that fixes nothing. Measured on the ten issues, Kedar's first Kaarigar: 8 fixed, 88 steps, 4 broken, plus everything it did that no test can see. The Kaarigar you finish in chapter 19 fixes 10 of 10, breaks nothing, refuses every risky command, and stops MD-111 (which has no cause in the code) with a note for a person. The interview task scores your loop on three issues.

Your turn

Make a checkout and run its tests in task 1. Write Kaarigar's loop in task 2 and fix MD-101 with it. Print the trail of a run in task 3. In the interview question, give it three numbers.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    the_repo.py: the repository and an issue

    Before Kaarigar fixes anything, look at what it will work on. Make a checkout with MD-101's bug in it (checkout("MD-101"), from workbench), run its tests with run_tests(root), and print, in this order:

    MD-101: TITLE (the issue's title, from issue("MD-101").title)

    tests: P passed, F failed

    then one line per failing test, as ✖ NAME (from the run's failing list).

  2. Task 2

    first_agent.py: Kaarigar's loop

    The tools are given in first_agent.py (SYSTEM, TOOLS, run_tool and AgentRun). Write run_agent(root, task, max_steps=10): send the system prompt, the task and TOOLS to lab-chat; while the model asks for tools, add its reply to the messages as a dict, run each call with run_tool(root, name, args) and add one tool message per call (with the call's id); ask again. Stop when it answers in text, or after max_steps requests.

    Return AgentRun(answer, steps, stop, messages): steps is the number of requests sent, stop is "answered" or "max_steps" (answer "" then).

    Under if __name__ == "__main__":, run it on a checkout of MD-101 with issue_text("MD-101"), print the answer, then steps: N, stop: STOP, then MD-101: fixed (or MD-101: not fixed (REASON)) from grade(root, "MD-101").

  3. Task 3

    trail.py: what did it touch?

    Jaya wants to know what an agent did before she trusts it. Run run_agent (from first_agent) on a checkout of MD-108 with issue_text("MD-108"). Then go through run.messages and print one line per tool call the model asked for: the request it came in (1 for the first), the tool's name and its path argument (or - when it has none), two spaces apart, like 1 read_file meterdown/night.py.

    Then print files written: PATHS (the paths write_file was called with, each once, comma and space between; none if there were none), and last requests: N (run.steps).

  4. Task 4 · interview question

    Interview: scorecard.py, is it any good?

    "Is your agent any good?" needs three numbers, not one. For MD-101, MD-102 and MD-107, in that order, each in a fresh checkout, run run_agent with the issue's text and grade it. Print one line per issue, two spaces apart:

    MD-101 fixed N steps B broken (not fixed when it was not; B is the length of the grade's broken list)

    Then print:

    Fixed: F/3

    Steps: S (all three runs' steps added up)

    Broken: B (all the broken tests added up)

What you will learn

  • MeterDown's repository, its tests and its issues, and the workbench that makes a fresh copy for every run.
  • What a model does on its own with an issue, and why that is not work.
  • The coding agent's loop: the model asks for a tool, your code runs it, the result goes back, until it answers.
  • The trail: what the agent touched, request by request.
  • Three numbers: issues fixed, steps taken, tests broken.

More in The tools

Build "Meet MeterDown, and Kaarigar's first loop" yourself

MeterDown, a Pune app for auto-rickshaw fares, has eleven open issues. You build Kaarigar, an agent that fixes them without breaking tests, running risky commands or touching files outside the repository. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

Free with a Google or GitHub sign-in, like the first 2 chapters of every lab.