Pydantic AI · Chapter 20 of 23

An agent behind an API

By the end of this chapter you can put an agent behind a web API with FastAPI, answer a farmer's question on a POST route, stream the answer as it is written with server-sent events, give every request its own farmer as deps, and explain why one agent object serves every farmer at once. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Production
  • 4 tasks
  • Python
  • In the all labs plan

The first 2 chapters of Pydantic AI are free. This one comes with the all labs plan.

Why you need this

The assistant has lived in scripts you run in a terminal. Farmers do not have your terminal. KhetBazaar's WhatsApp service, the app and Ganesh's tablet all need to reach it the same way: by sending an HTTP request and getting an answer back. Pradnya wants all three live by the season's first onion harvest.

An API is a program that waits for requests and answers each one. Many farmers write at the same moment, so it answers many at once, and each answer must be about the farmer who asked. That last part is where an agent behind an API usually goes wrong.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    api.py: a POST route for the agent

    Make a FastAPI app named app, and an agent for openai-chat:lab-chat with the instructions You are KhetBazaar's farmer assistant. Answer in one or two sentences. made once at the top of the file.

    POST /ask takes a JSON body with one field, message (1 to 500 characters; anything else answers 422 without asking the model). An async route awaits the agent's run on the message and answers {"reply": the output, "input_tokens": N, "output_tokens": N}, the tokens from the run's usage.

  2. Task 2

    stream.py: stream the answer

    Make an app with POST /ask/stream, the same body as task 1, and an agent for openai-chat:lab-chat with the instructions You are KhetBazaar's farmer assistant. The route streams the run's text as server-sent events (text/event-stream): one event per new piece of text, data: {"text": "<the piece>"} followed by a blank line, then a last event, data: [DONE].

    The check joins your pieces and compares them with what the model streamed, and refuses an empty message with 422 before asking the model.

  3. Task 3

    deps.py: each request's farmer

    POST /ask takes {"farmer_id": ..., "message": ...} (message 1 to 500 characters). For an id that is not a farmer, answer 404 without asking the model. Otherwise make the farmer's deps from khetbazaar.get_farmer and run the agent with them, answering {"reply": the output}.

    The agent, made once, has instructions that start You are KhetBazaar's farmer assistant. Use the tools. and dynamic instructions from the deps naming the farmer's name and asking for a reply in their language; a tool credit_balance that answers khetbazaar.credit_balance for the farmer in the deps and takes no parameters; and a tool mandi_price(crop, mandi). The check asks for two farmers how much credit they have left.

  4. Task 4 · interview question

    Interview: shared.py, one agent for every request

    shared.py holds an API that was right in testing and told Sunita Vitthal's credit balance on Monday morning. Fix it: make the agent once, at the top level of the file, and pass each request's farmer to the run, so that the instructions (The farmer is <name>.) and the credit_balance tool read it from the run, never from a global. POST /ask with {"farmer_id": ..., "message": ...} still answers {"reply": the output}.

    The check sends three farmers' questions at the same moment. The interviewer will ask: why one agent object for every request, and where does each request's state live?

What you will learn

  • FastAPI in brief: an app, a route, a body checked by Pydantic.
  • An async route that awaits the agent.
  • Streaming an answer as server-sent events.
  • Deps per request: the farmer who asked, in the run.
  • One agent for every request, and where each request's state lives.

More in Production

Build "An agent behind an API" yourself

KhetBazaar, a Nashik agritech startup, answers farmers on WhatsApp from a bike. You build its farmer assistant. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Pydantic AI are free. This one comes with the all labs plan.