Memory and multi-agent systems · Chapter 13 of 16

Parallel agents

By the end of this chapter you can ask several specialists at the same time with AsyncOpenAI and asyncio.gather, give each one a time limit, merge their answers into one reply a family can read, and stay under a provider's rate limit with a semaphore and polite retries. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Many agents
  • 5 tasks
  • Python
  • In the all labs plan

The first 2 chapters of Memory and multi-agent systems are free. This one comes with the all labs plan.

Why you need this

Some questions belong to two specialists. Twelve of the 132 client requests in requests.jsonl say so in their needs: "Can we afford Amer Bagh Palace if we cut the decor?" needs the venue and the budget, "Is the haveli's kitchen big enough for the caterer?" the venue and catering. Policy 6.3 is short: a request that needs two specialists gets both answers, merged into one reply.

Sameer timed the first try. It asked one specialist, waited, then asked the other, and a family waited for both. Karan put it simply: the two specialists do not need each other, so they should not wait for each other. And Rekha added the part nobody had thought of: "The family gets one message. Not two messages, and certainly not the same sentence twice."

What you build: 5 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    parallel.py: specialists at once

    Write ask(name, text, wedding_id, model="lab-chat") in parallel.py: one specialist's answer with the async client at temperature 0, sending the specialist's system(wedding_id) from specialists.py as the system message and the text as the user message. Write ask_all(text, wedding_id, names, model="lab-chat", timeout=10): ask every named specialist at the same time and return a dict, name to answer, in the order of names; a specialist that has not answered after timeout seconds gives None.

    Under if __name__ == "__main__":, ask venue and decor about WD1063, Can the photographer shoot the haldi at the venue?, on lab-chat-slow, first one after the other, then with ask_all, and print two lines, one after the other: then the seconds with one decimal and s, and at once: the same way. The check times ask_all on lab-chat-slow with three specialists, and with a timeout of 1 second.

  2. Task 2

    needs.py: the requests that need two

    For each request of requests.jsonl whose needs names two specialists, in the file's order, ask both at once with ask_all, and print a line with the request's id, a space, its needs joined with a comma and a space, a colon, a space and its text; then for each specialist in the order of needs, two spaces, its name, a colon, a space and its answer.

    End with the line the same answer from both: then how many of those requests got the very same answer from both specialists, of, and how many there were.

  3. Task 3

    merge.py: one reply from several answers

    Write merge(answers, needs) in merge.py: answers maps a specialist to its answer or None; needs is the order. Return one reply with one line per different answer, in the order of needs: the names that gave it joined with and and a space on each side, the first letter a capital, a colon, a space and the answer. Answers equal after stripping spaces at the ends are the same answer. A None or empty answer gets no line; when there are any, the last line is No answer yet from: and those names joined with a comma and a space, then a full stop.

    Under if __name__ == "__main__":, for R031 and R055, ask both specialists with ask_all and print the request's id, a colon, a space and its text, then merge's reply.

  4. Task 4

    limits.py: forty questions, three at a time

    Make the AsyncOpenAI client in limits.py with max_retries=0. Write ask_many(questions, limit=3, model="lab-chat-limited"): each question is a tuple of a specialist, a text and a wedding id; ask each like task 1's ask, never more than limit at a time, and return the answers in the order of the questions. A 429 is retried after the seconds in its retry-after header, up to 4 retries.

    Under if __name__ == "__main__":, ask the first 40 requests of requests.jsonl, each to its route specialist, and print the number of answers, answers in, the seconds with one decimal, s, a comma, a space, the number of 429s your requests met and refused on the way. The check runs ask_many with limit=6 too, which lab-chat-limited refuses.

  5. Task 5 · interview question

    Interview: fix_me.py, parallel agents that wait in line

    ask_all(text, wedding_id, names) in fix_me.py should ask every named specialist at the same time. Run it: it asks them one after the other, and a name that is not a specialist loses every answer.

    Fix ask_all so that the specialists are asked at the same time and a specialist that fails answers None, while the others' answers are kept, in a dict in the order of names. Keep the function's name and parameters. The check times it on lab-chat-slow and gives it a specialist that does not exist.

What you will learn

  • Async requests: AsyncOpenAI, await, and asyncio.gather.
  • A time limit per agent, so one slow specialist does not hold up the reply.
  • Merging answers into one reply, without repeating anything.
  • Rate limits: a semaphore, the 429 status, and retrying after retry-after.

More in Many agents

Build "Parallel agents" yourself

Baraat Bliss, a Jaipur wedding planner, must remember months of family decisions across many agents. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Memory and multi-agent systems are free. This one comes with the all labs plan.