OpenAI Agents SDK · Chapter 18 of 24
Tracing
By the end of this chapter you can say what a trace and a span are, switch the SDK's tracing on and give it a processor of your own, read the spans of a run (the agent, each response, each tool), find what made a run slow, put several runs of one conversation in one trace, add a span of your own, and keep a parent's numbers out of every trace. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Memory
- 4 tasks
- Python
- In the all labs plan
The first 2 chapters of OpenAI Agents SDK are free. This one comes with the all labs plan.
Why you need this
Bhanu forwards a complaint: "the bot took ages to answer about the timetable". Nidhi has another: a parent says the assistant gave a wrong refund on Tuesday. Neither run is on anyone's screen any more. What did the model get, which tools ran, how long did each part take?
lab requests shows the requests the lab model received, but only in your lab, and it says nothing about your own code: how long a tool took, which agent was running, which runs belong to one parent's chat. A helpdesk that talks to hundreds of families needs a record of every run, kept by the program itself. The Agents SDK writes that record as it runs: it is called a trace.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
spans.py: a run's spans, as they end
Write a tracing processor (a subclass of TracingProcessor from agents.tracing, with all six methods) whose on_span_end prints one line per span: agent: and the agent's name for an agent span, function: and the tool's name for a function span, response: N in, N out for a response span (its usage's input and output tokens), and the span's type alone for any other span.
Switch tracing on and make your processor the only one. Make an agent for lab-chat named Shikhar helpdesk, with the instructions You are Shikhar Classes' helpdesk assistant. Use the tools. and two function tools, refund_quote(student_id) and class_schedule(batch_id, day), returning json.dumps of shikhar's functions of the same names, each with a docstring. Run it on:
S2053 ka refund kitna milega? Aur NEET-28X ka kal ka timetable?
Print the run's final output after the spans' lines.
- Task 2
slow_tool.py: find the slow tool
Write slowest_tool(spans): given the finished spans of one run (the Span objects a processor's on_span_end receives, in any order), return a tuple of the name of the tool whose function span took longest and how long it took in milliseconds, rounded to a whole number: ("class_schedule", 412).
The Check passes it the spans of real runs where different tools are slow. The response, turn, agent and task spans of a run all take longer than any tool: compare only the function spans.
- Task 3
conversation.py: one trace for a conversation
Write a processor that counts, for each trace, its runs (task spans) and its requests (response spans), and in on_trace_end prints: Trace: the trace's name, a comma, group and the trace's group id, a colon, then N runs, N requests.
Make an agent for lab-chat with the instructions You are Shikhar Classes' fees desk. Use the tools. and two tools, fee_status(student_id) and refund_quote(student_id), returning json.dumps of shikhar's. In one trace named Parent chat with the group id S2061, run it on S2061 ki fees kitni baki hai? and print the final output; then run it again on the first result's input list with the user message Aur agar S2061 chhod de toh refund kitna milega? added, and print that final output.
Prints the two replies, then Trace: Parent chat, group S2061: 2 runs, N requests (your count).
- Task 4 · interview question
Interview: private.py, what never goes in a trace
Write a processor whose on_span_end prints each span's export() as JSON, one line per span. Make an agent for lab-chat with the instructions You are Shikhar Classes' fees desk. Use the tools. and a tool request_refund(student_id, reason) returning json.dumps of shikhar.request_refund. Run it on the message exactly:
S2072 ka refund start kar do. Account 31245678901, IFSC SBIN0004321, phone 9414012345.
with a run config that leaves sensitive data out of traces, and print the final output last. The Check reads the spans: they must keep the tool's name and every request's token counts, and hold none of the parent's numbers.
The interviewer will ask: what should a trace hold, and what should it never hold?
What you will learn
- Traces and spans: what the SDK records for every run.
- A processor of your own: switching tracing on in the lab and receiving every span.
- Reading spans: their type, their data, their start and end times.
- trace(): several runs as one trace, named and grouped, and custom_span for your own steps.
- Sensitive data: what a span holds, and how to leave it out.
More in Memory
Build "Tracing" yourself
Shikhar Classes, a Kota coaching institute, answers parents and students on WhatsApp all day. You build its helpdesk assistant. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of OpenAI Agents SDK are free. This one comes with the all labs plan.