Pydantic AI · Chapter 21 of 23

Durable conversations

By the end of this chapter you can keep a farmer's conversation in Postgres and continue it in another process, park a run that waits for a person's approval and finish it after a restart, make a tool safe to run twice, and say what a stored conversation needs when the server crashes halfway through a run. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Production
  • 4 tasks
  • Python
  • In the all labs plan

The first 2 chapters of Pydantic AI are free. This one comes with the all labs plan.

Why you need this

On Tuesday night KhetBazaar moved the API to a bigger server. The restart took eight seconds. On Wednesday morning Ganesh had three angry calls: farmers who had been talking to the assistant were greeted like strangers, and a hail claim that Sneha was about to approve had vanished. In chapter 6 you saved a conversation as JSON, and in chapter 12 a run stopped to wait for a person. Both lived in the process. A process ends: a deploy, a crash, a laptop lid closing.

What must outlive the process goes in a database. KhetBazaar's lab already runs Postgres, and this chapter puts conversations and waiting approvals in it. Then it looks at the harder part: what happens when the crash comes in the middle of a run.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    store.py: a conversation that survives a restart

    Take a farmer id and a message from the command line (python3 store.py F1008 "My message"). Connect to the Postgres database named by the environment variable DB_URI (postgresql:///khetbazaar when it is not set), and make the table conversations (farmer_id text primary key, messages jsonb) if it is missing.

    Load the farmer's stored messages (none the first time), run an agent for openai-chat:lab-chat with the instructions You are KhetBazaar's farmer assistant. on the message with that history, and save the run's whole conversation back in the farmer's row. Print the reply, then Messages stored: and how many messages the row now holds.

    The check runs it twice for one farmer, in two processes, with a database of its own, then once for another farmer.

  2. Task 2

    approval.py: approve tomorrow, in another process

    An agent for openai-chat:lab-chat (instructions You are KhetBazaar's farmer assistant. Use the tools.) has the farmer's id as deps, may output text or DeferredToolRequests, and has a tool file_claim(crop, cause, acres, loss_date) that needs approval and calls khetbazaar.file_claim for the farmer in the deps.

    python3 approval.py start F1008 "message" runs it; when the run waits, save its messages and the waiting tool call ids in a table approvals (one row per farmer) in the database from DB_URI, and print Waiting:, a space, the tool's name, a space and its arguments as JSON, one line per waiting call. python3 approval.py approve F1008 (or reject) loads the row, resumes the run with that decision for every waiting call, deletes the row and prints the reply; with nothing waiting it prints Nothing waiting for F1008 and asks no model. Each command runs in a new process.

  3. Task 3

    once.py: no double claims

    Write file_claim_once(tool_call_id, farmer_id, crop, cause, acres, loss_date): when a table tool_runs (tool_call_id text primary key, result jsonb) in the database from DB_URI already holds a result for this tool call id, return it and file nothing; otherwise call khetbazaar.file_claim, store its result under the id and return it.

    Then an agent for openai-chat:lab-chat (instructions You are KhetBazaar's farmer assistant. Use the tools.) with the farmer's id as deps and a tool file_claim(crop, cause, acres, loss_date) that calls file_claim_once with the run's tool call id. python3 once.py F1008 "message" runs it and prints the reply. The check calls your function twice with one id, again after KhetBazaar's memory is wiped as on a restart, and with a new id.

  4. Task 4 · interview question

    Interview: crash.py, what a crash leaves behind

    Write next_step(stored): stored is a conversation as the JSON that was saved (bytes or text, from the model messages type adapter). Return what a restart must do: ask the model when the last message is a request (the farmer's message, or tool results sent back); run the tools: and the tool names in order, joined by a comma and a space, when the last message is the model's response asking for tools (even with some text beside them); answered when it is a response with no tool calls; nothing to resume when there are no messages.

    The interviewer will ask: the server dies at each step of a farmer's claim; what happens, and what does the restart do?

What you will learn

  • Postgres from Python with psycopg, in a few lines.
  • A conversation in a row, as JSON, loaded and saved around each run.
  • A waiting approval in a row, decided by another process.
  • Idempotent tools: run twice, act once.
  • Reading a stored conversation after a crash.

More in Production

Build "Durable conversations" yourself

KhetBazaar, a Nashik agritech startup, answers farmers on WhatsApp from a bike. You build its farmer assistant. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Pydantic AI are free. This one comes with the all labs plan.