Agents from scratch · Chapter 12 of 18
Tracing
By the end of this chapter you can record what an agent did, step by step, in a file anyone can read later: a span per request and per tool call, a trace id per run, a summary of a run in five lines, and a replay of a run's tool calls without the model. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Agents that hold up
- 4 tasks
- Python
- In the all labs plan
The first 2 chapters of Agents from scratch are free. This one comes with the all labs plan.
Why you need this
A customer writes to Meenakshi at 6 pm: "your bot said it booked an electrician, nobody came." What happened at 11:04 that morning? The conversation is gone, the model will not remember, and running the request again today gives today's answer, from today's slots. lab steps showed you runs in the lab, but GharFix's server answers thousands of customers, and nobody is watching a terminal.
Agents fail in ways ordinary programs do not. A step can be "fine" (no exception, status 200) and still be the step where the run went wrong: a lookup that came back empty, a result missing a key the next tool needed, a model that answered instead of booking. To find that step afterwards, the run must have written down, as it went, what it sent, what came back, what it cost and how long it took. Vikram calls that a trace, and Farooq likes it for another reason: it is where the tokens are.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
tracer.py: a span per line
Write a class Tracer(path) in tracer.py. A new Tracer has a trace_id, a new random id as text, so that every run has its own. record(kind, **fields) adds one span to the file at path, as one line of JSON: an object with trace_id, span (the span's number in this trace, from 1), kind, and then every field given; it returns that object. Spans are added to the end of the file, so one file holds many traces.
Then write read_trace(path, trace_id=None): the spans in the file, in order, as dictionaries; given a trace_id, only that trace's.
- Task 2
traced_run.py: trace a booking
Write run_traced(question, tracer) in traced_run.py: chapter 3's loop with lab-chat, a system message saying today is 2026-10-22, the question, and GharFix's tools, at most 10 requests, returning the final answer. Record, with the tracer it is given:
after every request, a span of kind model with step (the request's number, from 1), model (the response's model), prompt_tokens, completion_tokens, seconds (how long the request took), finish_reason and tools (the names of the tools asked for, a list);
after every call, a span of kind tool with step (the request that asked for it), tool, arguments (the JSON text the model wrote), result_chars (the length of the result's JSON text sent back), error (the result's error text, or None) and seconds.
The check runs a booking and a reschedule GharFix refuses, and compares the spans with what the model received.
- Task 3
summary.py: a trace in five lines
Write summarize(path, trace_id) in summary.py. It reads one trace (use read_trace from tracer.py) and returns a list of lines:
trace <id>: <n> model calls, <n> tool calls
tokens: <total prompt> prompt, <total completion> completion
seconds: <the seconds of every span added up, with 2 decimals>
tools: <the tools called, in order, with a comma and a space between them, or none>
and then one line per tool span with an error, error at step <step>, <tool>: <error>, or, when there are none, errors: none.
The check writes a file with two traces in it and summarizes each.
- Task 4 · interview question
Interview: fix_from_trace.py, find the failing step
Farooq asked for smaller tool results, so the team wrapped find_technicians to keep only what the agent needs. Since then, bookings end without a booking, and no result is ever an error. fix_from_trace.py runs a booking and writes its trace to fix_from_trace.jsonl.
Read the trace, find the step where the run went wrong, and fix the tool so that the run books: for the booking in the file, and for one that names no slot. Keep the trim: find_technicians's rows still leave jobs_done out.
Keep run(question), find_technicians and FUNCTIONS. The check calls your find_technicians and runs both bookings.
What you will learn
- Spans: one record per model request and per tool call.
- A trace: a run's spans, tied together by a trace id, one JSON line each.
- Reading a trace: a summary, and finding the step that went wrong.
- Replaying a run's tool calls from its trace, without the model.
More in Agents that hold up
Build "Tracing" yourself
GharFix, a Bengaluru home-services company, needs an assistant that books, reschedules and refunds safely. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of Agents from scratch are free. This one comes with the all labs plan.