Pydantic AI · Chapter 7 of 23

Streaming

By the end of this chapter you can run an agent with async code, answer several farmers at the same time, stream a reply so the words appear as the model writes them, stream a typed output as it fills, and say when streaming helps and when it does not. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Real-world agents
  • 4 tasks
  • Python
  • In the all labs plan

The first 2 chapters of Pydantic AI are free. This one comes with the all labs plan.

Why you need this

KhetBazaar's app has a chat screen now, and Ganesh's team uses it all day. Pradnya watched a farmer use it at the Lasalgaon mandi: he typed his question, saw nothing for two seconds, decided it was broken and phoned Ganesh instead. "The answer was coming," she says. "He just could not see it coming."

Ganesh has the other problem. At 7 am, when the mandi opens, forty farmers ask for the rate at once. The assistant answers them one after another, and the fortieth farmer waits for the other thirty-nine.

Kedar has an answer for both: stop waiting for one thing at a time. That is what async is for, and Pydantic AI is built on it.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    stream_reply.py: a reply as it is written

    Pradnya wants the KhetBazaar app to show a reply while it is being written, not after a pause. Make an agent for openai-chat:lab-chat with the instructions You are KhetBazaar's farmer assistant. and stream its reply to the farmer's question: When will I get paid for my grapes?

    Print each new piece of text as it arrives, on one line with nothing between the pieces, and end that line when the reply is done. Then print a second line, Tokens out: and the run's output tokens from the result's usage.

  2. Task 2

    stream_sale.py: a typed object as it fills

    Ganesh's screen should fill in a sell request as the model writes it. Make a Pydantic model SellRequest with crop (one of onion, grape, tomato, pomegranate), quantity (a float: how many quintals the farmer wants to sell), mandi (a string or None, default None) and graded (True, False or None, default None). Make an agent for openai-chat:lab-chat with SellRequest as its output type.

    Stream the run on the farmer's message: 35 quintal graded tomato Pimpalgaon la vikaycha ahe

    Print every partial SellRequest as it arrives, without waiting between them (debounce_by=None), one per line as JSON (model_dump_json). The last line is the whole request.

  3. Task 3

    several_farmers.py: four farmers at once

    At 7 am, after the mandi opens, many farmers write at once. lab-chat-slow takes about 4 seconds per answer. Make one agent for openai-chat:lab-chat-slow with the instructions You are KhetBazaar's farmer assistant. Answer in one sentence. and ask it these four questions at the same time:

    Is there any fee when I sell pomegranate?

    Can I buy fertiliser on credit?

    Will it rain in Sinnar this week?

    Can I return an opened packet of seed?

    Print the four replies, one per line, in the questions' order. One after another they take 16 seconds; together, about 4.

  4. Task 4 · interview question

    Interview: when_to_stream.py, the first piece or the whole reply

    Make an agent for openai-chat:lab-chat with the instructions You are KhetBazaar's farmer assistant. Stream its reply to: What spray should I use for thrips on my onion?

    Time it with time.perf_counter: the seconds until the first piece of text arrived, and the seconds until the stream ended. Print two lines, each with two decimals: First piece: 0.05s, then Whole reply: 0.40s (your numbers).

    The interviewer will ask: KhetBazaar's farmers read the answers on WhatsApp, and Ganesh's team in the app. Where does streaming help, where does it not, and when must you wait for the whole output before acting on it?

What you will learn

  • Async in two minutes: async def, await, asyncio.run, and the agent's run, the async twin of run_sync.
  • Several farmers at once with asyncio.gather.
  • Streaming a reply with run_stream and stream_text.
  • Streaming a typed output with stream_output, as it fills.
  • When to stream, and when to wait for the whole answer.

More in Real-world agents

Build "Streaming" yourself

KhetBazaar, a Nashik agritech startup, answers farmers on WhatsApp from a bike. You build its farmer assistant. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Pydantic AI are free. This one comes with the all labs plan.