Pydantic AI · Chapter 5 of 23

Validation and retries

By the end of this chapter you can say what Pydantic AI checks in everything a model sends back (tool arguments and outputs), what it does when a check fails (a retry prompt the model reads), raise ModelRetry from your own tools and output validators, set a run's retry budget, and handle UnexpectedModelBehavior when the budget runs out. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Your first agents
  • 4 tasks
  • Python
  • In the all labs plan

The first 2 chapters of Pydantic AI are free. This one comes with the all labs plan.

Why you need this

Ganesh has seen it all on WhatsApp: a farmer who types 4000 quintals for a 40 quintal lot, a claim number with one digit wrong, a farmer id that does not exist. Models make the same kinds of slips: an argument under the wrong name, JSON with a stray comma, a number nobody could grow on two acres. Sneha will not book any of it.

In chapters 2 to 4 everything went right. This chapter is about the other days. Pydantic AI already validates everything a model sends; here you see what happens when validation fails, and add KhetBazaar's own rules on top.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    retry_tool.py: a tool that asks the model to retry

    Give an agent for openai-chat:lab-chat, with the instructions You are KhetBazaar's farmer assistant., a tool named claim_status, with a docstring, that takes a claim_id and returns khetbazaar.claim_status for it. KhetBazaar's function raises a ValueError for a claim that does not exist: catch it in the tool and raise Pydantic AI's ModelRetry with the ValueError's message instead.

    Run the agent on the farmer's question exactly: What is the status of my claim KC4999?

    and print the output. The run must not stop with an error.

  2. Task 2

    sale_validator.py: an output validator

    Write a Pydantic model Sale with crop (str) and quantity (float), and an agent for openai-chat:lab-chat with Sale as its output type. Add an output validator: when the quantity is not more than 0 and at most 200 quintals, it raises ModelRetry with the message: A lot must be more than 0 and at most 200 quintals. Otherwise it returns the sale.

    Run the agent on two messages, in order: I have 40 quintal onion graded, want to sell this week and 4000 quintal kanda tayar ahe. For a sale, print crop, quantity quintals (for example: tomato, 12.0 quintals). When the run gives up with UnexpectedModelBehavior, catch it and print Rejected: and the error.

  3. Task 3

    retry_budget.py: a retry budget

    lab-chat-sloppy wraps its JSON in a code fence with a trailing comma, until it is told the JSON is invalid. Write a Pydantic model Sale (crop as str, quantity as float) and make ONE agent for openai-chat:lab-chat-sloppy whose output type is Sale in prompted mode.

    Run it twice on 6 quintal anar tayar ahe, choosing the retry budget for each run: first 0 retries, then 1. For each run print retries=N: then either the sale and the requests, as crop, quantity quintals in R requests, or, when it raises UnexpectedModelBehavior, the error.

  4. Task 4 · interview question

    Interview: who_sees.py, who sees the error?

    Make two agents for openai-chat:lab-chat, each with a tool named claim_status that takes a claim_id and returns khetbazaar.claim_status. The first agent's tool turns KhetBazaar's ValueError into ModelRetry, as in task 1; the second agent's tool does not catch it.

    Run both on Has my claim KC4321 been paid? Print With ModelRetry: and the first run's output. Run the second inside a try block and print Without: then the exception's type name, a colon and a space, and the exception.

    The interviewer will ask who sees the error message in each case, and why that changes what you write in it.

What you will learn

  • Validation of tool arguments and output, and the retry prompt the model gets when it fails.
  • ModelRetry: your tool or validator asks the model to try again.
  • @agent.output_validator: your own rules on the output.
  • retries and UnexpectedModelBehavior: the budget, and what happens when it runs out.

More in Your first agents

Build "Validation and retries" yourself

KhetBazaar, a Nashik agritech startup, answers farmers on WhatsApp from a bike. You build its farmer assistant. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Pydantic AI are free. This one comes with the all labs plan.