Agents from scratch · Chapter 4 of 18

Parallel calls

By the end of this chapter you can handle a reply that asks for several tools at once: run every call, send one result per call id in any order, see when the model fans a lookup out, and turn parallel calls off when you need one call at a time. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: The loop
  • 4 tasks
  • Python
  • In the all labs plan

The first 2 chapters of Agents from scratch are free. This one comes with the all labs plan.

Why you need this

Customers do not ask one thing per message. "Cancel GF825369 and GF318075 please." "How much is it, and who can come to Koramangala?" Meenakshi sees messages like these all day, and a booking agent that handles them one request at a time is slow and expensive: every request resends the whole conversation.

Models know this. Asked about two bookings, a model asks for two tool calls in the same reply, and expects both results back before it says anything. Your loop from chapter 3 has to run all of them and pair each result with the call it answers. A loop that runs only the first call works for every customer who asks one thing at a time, which is exactly why nobody notices it until a customer asks two.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    run_all.py: two questions, two tools

    Send lab-chat a system message (GharFix's booking assistant; it must say today is 2026-10-22), the user's message, exactly:

    How much is an AC service, and who can come to Whitefield?

    and GharFix's tools (gharfix.TOOLS). The model asks for two tools in one reply. For every call in that reply, in order, print one line: the call's id, a space, the tool's name, a space, and its arguments exactly as the model wrote them. Run each call with GharFix's function and append its tool message.

    Then send the second request and print only the text of the model's final answer.

  2. Task 2

    out_of_order.py: results in any order

    Send lab-chat the same kind of system message, the user's message, exactly:

    Cancel GF424921 and GF186543 please.

    and GharFix's tools. The model asks for cancel_booking twice in one reply. Run the calls last first: the reply's last call first, then the one before it. As you run each one, print one line: the call's id, a space, the booking ID from its result, a space, the word refund, a space, and the refund from its result. Append each call's tool message in the order you run them.

    Then send the second request and print only the text of the model's final answer.

  3. Task 3

    fan_out.py: count the calls per step

    Run this request through the loop with the same kind of system message and GharFix's tools, exactly:

    Book an electrician in BTM Layout on 27 Oct: a switch is sparking. I am Rekha Nair, phone 9741234567.

    Run it twice: first as it is, then with parallel_tool_calls set to False in every request, calling gharfix.reset() between the two runs. For each run, print one line:

    parallel on: 1, 2, 1 calls, 4 requests, 5100 prompt tokens

    (illustrative numbers; the second line starts with parallel off:): the number of calls in each reply that asked for tools, in order, joined with a comma and a space; then the number of requests; then the prompt tokens of all the run's requests added up.

  4. Task 4 · interview question

    Interview: fix_first_only.py, the loop that answers one call

    fix_first_only.py's agent worked for every customer who asked about one thing. This customer asks about two bookings, and the API refuses the loop's second request. Fix the loop so that every call in a reply gets its own tool message, with its own id and result, before the next request.

    Keep run_agent(question), the system message, the customer's message and the one answer it prints.

What you will learn

  • Several calls in one reply, each with its own id.
  • One tool message per call, in any order: the id does the pairing.
  • Fanning out: a lookup asked for once per row of an earlier result.
  • parallel_tool_calls=False, and what it costs or saves.

More in The loop

Build "Parallel calls" yourself

GharFix, a Bengaluru home-services company, needs an assistant that books, reschedules and refunds safely. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Agents from scratch are free. This one comes with the all labs plan.