Agents from scratch · Chapter 14 of 18

Async tools

By the end of this chapter you can write the agent loop with async and await: serve many customers from one program, run a reply's independent tool calls at the same time, and put a time limit on every tool and every model request. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Production
  • 4 tasks
  • Python
  • In the all labs plan

The first 2 chapters of Agents from scratch are free. This one comes with the all labs plan.

Why you need this

GharFix's bot is one Python program answering every customer on WhatsApp. At 9 in the morning that is dozens of conversations at once, and each booking is several requests to the model, a second or so each. A loop that waits for one customer's reply before it even reads the next customer's message makes everyone wait in a single queue.

Inside one run there is waiting too. Vikram's team moved the technicians' calendar to a service of its own, and a check_slots call now takes about a second over the network. A booking with no slot asks for it once per technician, three calls in one reply, and they wait for each other for no reason. Then one morning the calendar service hangs, and every customer who asks for a slot waits with it, forever. Meenakshi gets the complaints.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    async_loop.py: an async loop

    GharFix's server answers many customers at once. Write run_agent(question) in async_loop.py as an async function: chapter 3's loop with lab-chat, a system message saying today is 2026-10-22, the question and GharFix's tools, at most 10 requests, returning the final answer's text.

    Talk to the model with AsyncOpenAI and await each request. GharFix's functions are plain functions: run each call with its function from gharfix.FUNCTIONS in a thread, and await it.

    The check runs a booking and a reschedule one after the other, then the two at the same time, and watches which thread each tool runs in.

  2. Task 2

    at_once.py: independent calls at once

    GharFix's calendar is now a service of its own: check_slots in at_once.py takes about a second, and a booking with no slot asks for it once per technician, in one reply. run_tool(name, arguments) is written for you.

    Write run_calls(tool_calls), async: run every call of one reply at the same time and return a list with one tool message per call (role tool, the call's id as tool_call_id, the result as JSON text as content), in the calls' order. Then run_agent(question), async: the loop with lab-chat, a system message saying today is 2026-10-22, the question and GharFix's tools, at most 10 requests, using run_calls; after each step's calls, print step N: K calls in S.SS s, and return the final answer.

    The check times run_calls on three check_slots calls and on a mix of tools, and runs a booking with no slot. Keep check_slots as it is.

  3. Task 3

    timeouts.py: a timeout per tool, and for the model

    Write run_tool(name, arguments, seconds=2) in timeouts.py, async: run FUNCTIONS[name] (GharFix's tools as async functions, given in the file) with the arguments, the model's JSON text, and return the result as JSON text. When the tool has not answered after seconds, stop waiting and return JSON text of an object with an error key whose text names the tool.

    Then run_agent(question, model="lab-chat"), async: the loop with that model, a system message saying today is 2026-10-22, the question and GharFix's tools, at most 10 requests, every call run with run_tool. Give each model request at most 3 seconds and no retries; when a request takes longer, return SORRY (given in the file), word for word. Otherwise return the final answer.

    The check swaps in a get_booking that never answers, and asks lab-chat-slow, which takes 4 seconds.

  4. Task 4 · interview question

    Interview: fix_serial.py, calls that wait in line

    In fix_serial.py, a booking with no slot fans out into three check_slots calls in one reply, and the loop is meant to run them at the same time. The step still takes about three seconds. Fix it so that every call of one reply runs at the same time: three check_slots calls in about 1 second.

    Keep check_slots exactly as it is: it stands for the calendar's old client library, which blocks while it waits, and which you cannot change. Keep run_calls(tool_calls), which returns one tool message per call in the calls' order, and run_agent(question).

    The check times run_calls on three check_slots calls and runs a booking.

What you will learn

  • An async loop: AsyncOpenAI, await, and running GharFix's plain functions in a thread.
  • Many customers at once with asyncio.gather.
  • A reply's calls at the same time, results kept in order.
  • Timeouts: a tool that does not answer, and a model that does not either, each turned into something the run can handle.

More in Production

Build "Async tools" yourself

GharFix, a Bengaluru home-services company, needs an assistant that books, reschedules and refunds safely. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Agents from scratch are free. This one comes with the all labs plan.