Pydantic AI · Chapter 8 of 23

Limits and settings

By the end of this chapter you can stop an agent run that will not end, with a limit on requests, tool calls or tokens; read what a stopped run spent; send settings such as temperature and max_tokens with every request or with one run; and say what limits a real assistant needs. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Real-world agents
  • 4 tasks
  • Python
  • In the all labs plan

The first 2 chapters of Pydantic AI are free. This one comes with the all labs plan.

Why you need this

Sneha opens the provider's bill on Monday and goes quiet. One test run on Friday evening made hundreds of requests. Nobody asked it to; a tool kept returning something the model did not like, so the model asked for it again, and again, and the agent obliged.

Kedar knows the pattern. "An agent is a loop," he says. "The model decides when it ends. So we decide when it must end." Pydantic AI has a ceiling for that, and a place for every other setting a request carries.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    request_limit.py: a ceiling on requests

    lab-chat-loopy asks for the same tool again after every result, forever. Make an agent for openai-chat:lab-chat-loopy with the instructions You are KhetBazaar's farmer assistant. and a tool mandi_price(crop: str, mandi: str) -> dict, with the docstring Today's price of a crop at a mandi, in rupees per quintal., that returns khetbazaar.mandi_price(crop, mandi).

    Run it on What is the tomato price in Pimpalgaon today? with a limit of 4 requests. The limit stops the run with an error: catch it and print one line, Stopped: and the error.

  2. Task 2

    token_budget.py: a token budget, and what was spent

    Make the same agent and tool as task 1. Run it on What is the grape price in Nashik today? with a limit of 1500 total tokens, and pass the run a RunUsage of your own so you can read it after the run stops.

    Catch the error, then print one line from your RunUsage: Spent: and its total tokens, then tokens (like Spent: 1700 tokens).

  3. Task 3

    settings.py: settings for the agent, and for one run

    Make an agent for openai-chat:lab-chat with the instructions You are KhetBazaar's farmer assistant. and model settings with a temperature of 0, so its answers are repeatable.

    Run it twice on Can I buy seeds on credit? The first run uses the agent's settings as they are; the second also sets max_tokens to 10, for that run only. Print two lines: Full: and the first output, then Short: and the second.

  4. Task 4 · interview question

    Interview: fortune.py, the loop that would have cost a fortune

    Make the agent and tool of task 1 again, and run it on What is the onion price in Lasalgaon today? with a limit of 10 requests and a RunUsage of your own. Catch the error.

    Print three lines from your RunUsage: Requests: N, then Tokens: N in, N out, then Cost: ₹ and the run's cost in rupees with two decimals. lab-chat costs ₹200 per million input tokens and ₹800 per million output tokens.

    The interviewer will ask: Pydantic AI stops a run at 50 requests unless you say otherwise. If this bug shipped and 2,000 farmers wrote a day, what would a month cost, and what limits would you set for KhetBazaar's assistant?

What you will learn

  • Why a run can loop, and what each turn of the loop costs.
  • UsageLimits: requests, tool calls and tokens, and UsageLimitExceeded when one is reached.
  • Reading what a stopped run spent, with your own RunUsage.
  • ModelSettings: temperature, max_tokens and timeout, for every run of an agent or for one run.

More in Real-world agents

Build "Limits and settings" yourself

KhetBazaar, a Nashik agritech startup, answers farmers on WhatsApp from a bike. You build its farmer assistant. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Pydantic AI are free. This one comes with the all labs plan.