Agents from scratch · Chapter 13 of 18

Streaming agents

By the end of this chapter you can run an agent with stream=True on every step: put streamed tool calls back together piece by piece, run them, print the final answer as it is written, and still get the token counts of every step. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Production
  • 4 tasks
  • Python
  • In the all labs plan

The first 2 chapters of Agents from scratch are free. This one comes with the all labs plan.

Why you need this

Anjali booked an AC technician through the bot on her own phone, and then stared at an empty chat for three seconds. "Every app I use shows the answer appearing. Ours looks dead, and then dumps a paragraph." She is right about the three seconds: a booking is three requests, and a request that is not streamed sends nothing until its last token is written.

You may have streamed a chat reply before: text arrives in pieces and you print each one. An agent is harder, because most of its replies are not text. They are tool calls, and a streamed tool call arrives in pieces too: its id and name in one chunk, its arguments a few characters at a time over many more. Your loop cannot run book_slot on half its arguments. Vikram wants a loop that streams every step, puts each call back together exactly, and shows the customer the answer the moment it starts.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    assemble.py: put a streamed reply back together

    Write read_stream(stream, on_text=None) in assemble.py. stream is what a request with stream=True returns (anything you can loop over that gives chunks). Read it to the end and return a tuple: the assistant's message as a dictionary, ready to append to the conversation, and the finish reason.

    The message has role assistant; content, the whole text joined from its pieces, or None when the reply has no text; and, only when the model asked for tools, tool_calls: a list with one entry per call, in the order of their index, each with its id, type function, and a function with the call's name and its whole arguments text. When on_text is given, call it with each piece of text the moment it arrives.

    The check reads lab-chat's streams (two calls, two tools, a text answer, one with a usage chunk at the end) and a stream whose two calls arrive mixed together.

  2. Task 2

    stream_loop.py: an agent that streams

    Write run_streaming(question) in stream_loop.py: chapter 3's loop with lab-chat, a system message saying today is 2026-10-22, the question, and GharFix's tools, with stream=True on every request (use read_stream from assemble.py), at most 10 requests.

    Before running each tool call, print one line with the tool's name in square brackets, like [get_quote]. Print the final answer's text piece by piece as it arrives, on one line, then end the line, and return the answer.

    The check runs a booking with a slot and one without, and watches how the answer reaches the screen.

  3. Task 3

    stream_usage.py: usage from the last chunk

    Farooq needs the token counts of streamed runs too. Write run_counted(question) in stream_usage.py: the same streaming loop (lab-chat, a system message saying today is 2026-10-22, the question, GharFix's tools, at most 10 requests), asking every request to include its usage. Return a tuple: the final answer's text, then the prompt tokens and the completion tokens of the whole run, added up over every request.

    The check runs a reschedule and a booking, and adds up the usage lab-chat reported.

  4. Task 4 · interview question

    Interview: fix_stream.py, the reader that loses a piece

    GharFix is adding a backup provider. Its reply to a customer who cancels two bookings, recorded chunk by chunk, is BACKUP in fix_stream.py, and the file's read_stream, which reads lab-chat's streams well, breaks on it. Run the file and read what it prints.

    Fix read_stream so that it puts any streamed reply back together, whichever provider sent it: every call with its id, its name and all of its arguments. Keep read_stream(stream), what it returns, and BACKUP.

    The check reads BACKUP, lab-chat's streams, a stream that sends each call whole in one chunk, and three calls streamed at once.

What you will learn

  • A streamed tool call: what each chunk carries, and the index that ties the pieces of one call together.
  • Putting a reply back together into the message your loop appends.
  • The streaming loop: what changes in the loop, and what the customer sees while the tools run.
  • Usage on the last chunk, for every step of a run.

More in Production

Build "Streaming agents" yourself

GharFix, a Bengaluru home-services company, needs an assistant that books, reschedules and refunds safely. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Agents from scratch are free. This one comes with the all labs plan.