AI booking agent project (TypeScript) · Chapter 9 of 12

Tracing

By the end of this chapter every step your agent takes is written to Postgres the moment it happens: each model call with what it asked for, its tokens and its time, and each tool call with its arguments, its result and its error. A member reads their own trace, Asha reads anyone's, and the totals say how many calls, tokens and milliseconds a conversation cost. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Ready for members
  • 3 tasks
  • TypeScript
  • In the all labs plan

The first 2 chapters of AI booking agent project (TypeScript) are free. This one comes with the all labs plan.

Why you need this

In the trial weekend, when a booking went wrong, the question was always the same: did the member ask for the 6:15 pm show, or did someone type 6:15 instead of 6:45? Asha had WhatsApp on one screen and the API's log on the other, and for some of the 17 wrong bookings nobody could tell.

With the agent, the complaints will sound like "the agent booked the wrong show" or "it said there were no seats". The server log from the booking backend shows that POST /bookings answered 201 in 4 ms; it does not show which date the model wrote, what find_showtimes returned, or which showtime id the model then picked. Chapter 6 stores the messages, but not how long each step took, how many tokens it used, or which steps failed. Asha needs the run itself, step by step, and Datta needs its token count: a trace.

What you build: 3 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    tracing.ts: trace a model step

    In src/agent/tracing.ts, write Tracer.modelStep(reply, sent, ms): one row through this.add({...}) with kind 'model', name agentConfig.model, input { messages: sent }, output { tool_calls: [{ name, arguments: argumentsOf(call) }, ...] } when the reply has calls or { text: reply.text } when it does not, promptTokens and completionTokens from reply.usage, and ms.

    The Check traces a reply that asks for find_showtimes (4 messages sent, 810 and 52 tokens, 731 ms) and an answer, then reads agent_steps: two rows with the run's id and every column as above.

  2. Task 2

    tracing.ts: trace a tool step

    In src/agent/tracing.ts, write Tracer.toolStep(call, result, ms): one row with kind 'tool', name call.name, input argumentsOf(call), output result, error String(result.error) when result is an object with an error key (else null), and ms.

    The Check traces find_shows with a list as its result, book_seats with a showtime_cancelled error result and a call whose arguments are not JSON, and wants the arguments as JSON (or the text), the whole result as output, the error's text in error, and the ms.

  3. Task 3 · interview question

    Interview: tracing.ts, read the trace

    In src/agent/tracing.ts, write traceView(conversationId, user): when the conversation does not exist, or is not the user's own and the user is not an admin, throw notFound('There is no conversation <id>'). Return { conversationId, steps: [{ runId, kind, name, input, output, error, promptTokens, completionTokens, ms }, ...] in the order they were written (by id), totals: { modelCalls, toolCalls, toolErrors, promptTokens, completionTokens, ms } }, the token sums counting a missing count as 0.

    The Check runs Neha's booking through the app, then reads her trace: model, find_showtimes, model, book_seats, model, with the step keys above and totals equal to what the lab model really received (3 model calls, 2 tool calls, the tokens, the sum of the ms). Pranav gets a 404, conversation 999 a 404, Asha (admin) the whole trace; GET /agent/conversations/latest/trace answers 200.

    Nikhil will ask: Neha says the agent booked the wrong show. What do you look at in her trace, and in what order?

What you will learn

  • A trace: one row per step of a run, model or tool, in the order they happened.
  • Hooks: how the loop reports each step without knowing where the report goes.
  • What to keep for a step: inputs, outputs, errors, tokens and time, and what not to copy again.
  • Writing as you go: a run that fails half way still leaves its steps.
  • Reading it back: only the member's own trace, or an admin's view; totals from the same rows.

More in Ready for members

Build "Tracing" yourself

Parda members want to book by chat. You build the agent that finds the show, holds the seats through Parda's own API, and asks before it pays. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of AI booking agent project (TypeScript) are free. This one comes with the all labs plan.