OpenAI Agents SDK · Chapter 7 of 24
Streaming
By the end of this chapter you can run an agent with async code, answer several parents at the same time with asyncio.gather, stream a reply so the words appear as the model writes them, watch a tool being called while the run streams, and say when streaming helps and when it does not. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Several agents
- 4 tasks
- Python
- In the all labs plan
The first 2 chapters of OpenAI Agents SDK are free. This one comes with the all labs plan.
Why you need this
Shikhar's helpdesk has a chat screen in the student app now, beside WhatsApp. Bhanu watched a student use it in the corridor: she typed her question, saw nothing for two seconds, decided it was broken and walked to the counter instead. "The answer was coming," Bhanu says. "She just could not see it coming."
Kavita has the other problem. On a Tuesday at 6 pm, when the Sunday test results come out, a hundred parents write at once. The assistant answers them one after another, and the hundredth parent waits for the other ninety-nine.
Karthik has one answer for both: stop waiting for one thing at a time. Everything so far used Runner.run_sync, which waits with the whole program stopped. This chapter meets its async twin, and streaming.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
stream_reply.py: a reply as it is written
Bhanu wants the student app to show a reply while it is being written, not after a pause. Make an agent for lab-chat with the instructions You are Shikhar Classes' helpdesk assistant. and stream its reply to the student's question: How do I book a doubt session?
Print each new piece of text (each text delta) as it arrives, on one line with nothing between the pieces, and end that line when the stream is done. Then print a second line, Tokens out: and the run's output tokens from its usage.
- Task 2
stream_tools.py: a tool call while it streams
While the agent looks up the timetable, the app should say so. Make a function tool class_schedule(batch_id, day) that returns json.dumps of shikhar.class_schedule(batch_id, day), and an agent for lab-chat with the instructions You are Shikhar Classes' helpdesk assistant. Use the tools. and that tool. Stream its run on the student's message: what is the timetable for JEE-28X on Monday?
For each run item event print one line: when a tool is called, Calling and the tool's name, then with and its arguments exactly as the model sent them (Calling class_schedule with {"batch_id": ...}); when the tool's output comes back, Tool output: and the number of characters of that output, then characters. After the stream, print the run's final output on a last line.
- Task 3
three_parents.py: three parents at once
On result night many parents write at once, and lab-chat-slow takes about 4 seconds per answer. Make one agent for lab-chat-slow with the instructions You are Shikhar Classes' helpdesk assistant. Answer in one sentence. and ask it these three questions at the same time:
Do you have a library for students?
Is there a bus from the hostel to the classes?
Can parents visit on Sundays?
Print the three final outputs, one per line, in the questions' order. One after another they take 12 seconds; together, about 4.
- Task 4 · interview question
Interview: when_to_stream.py, the first piece or the whole reply
Make an agent for lab-chat with the instructions You are Shikhar Classes' helpdesk assistant. Stream its reply to: What should my daughter bring on the first day?
Time it with time.perf_counter: the seconds until the first text delta arrived, and the seconds until the stream ended. Print two lines, each with two decimals: First piece: 0.05s, then Whole reply: 0.40s (your numbers).
The interviewer will ask: parents read the answers on WhatsApp, students in the app. Where does streaming help, where does it not, and when must you wait for the whole reply before acting on it?
What you will learn
- Async in two minutes: async def, await, asyncio.run, and Runner.run, the async twin of run_sync.
- Several parents at once with asyncio.gather, and the one rule: never asyncio.run and run_sync in the same program.
- Streaming a reply with Runner.run_streamed and stream_events().
- Run item events: seeing a tool called while the run streams.
- When to stream, and when to wait for the whole answer.
More in Several agents
Build "Streaming" yourself
Shikhar Classes, a Kota coaching institute, answers parents and students on WhatsApp all day. You build its helpdesk assistant. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of OpenAI Agents SDK are free. This one comes with the all labs plan.