Memory and multi-agent systems in TypeScript · Chapter 14 of 16

When agents fail

By the end of this chapter you can stop two agents handing a conversation back and forth for ever, stop a model that calls the same tool again and again, give a conversation a budget in tokens and rupees, and let a referee choose between two specialists who disagree. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Many agents
  • 5 tasks
  • TypeScript
  • In the all labs plan

The first 2 chapters of Memory and multi-agent systems in TypeScript are free. This one comes with the all labs plan.

Why you need this

Last chapter's team works when every specialist knows its job. Rekha brought the morning after it did not. A family asked "How much will the Jain counter add to our budget?". Catering handed it to budget, budget handed it back to catering, and the logs showed them doing it all night. Sameer brought the bill for that night. And Pooja brought a worse one: a cousin had asked about the budget, and two specialists had given two different numbers, one of them January's.

None of these is a bug in one line of code. Each is what agents do when nothing stops them: a model has no sense of going round in circles, of time, or of money. The loops you have written so far end when the model answers, and nothing else ends them. Karan: "Every loop gets a limit, every limit has a fallback, and the fallback is a person."

What you build: 5 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    pingpong.ts: a hop limit

    Write and export run(text, weddingId, start, maxHops = 3) in pingpong.ts. The specialist start gets the text: a request to lab-chat at temperature 0 with its system(weddingId), the text as the user message, and its transferTools from tools.ts. When it calls transfer_to_ a specialist, that specialist gets the same text the same way. The first reply without a tool call is the answer. When a specialist asks for a handover after maxHops handovers, a planner takes it: the path ends with planner and the answer is PLANNER.

    run returns an array of the answer and the path: every agent the conversation reached, in order. Inside if (import.meta.main), run R091 from its route specialist and print the path joined with a space, an arrow made of a hyphen and a greater-than sign, and a space, then the answer.

  2. Task 2

    loopy.ts: a tool called again and again

    Write and export answer(question, model = 'lab-chat-loopy', maxSteps = 5) in loopy.ts: send the system message You are Baraat Bliss's planning assistant. and the question with tools: [RECALL] at temperature 0; while the reply has tool calls, run each with recall, add the assistant message and a tool message with each call's JSON result, and ask again. A call with the same name and arguments (compared as data) as one already run is not run: send the messages once more with tool_choice 'none' and return that reply. Do the same after maxSteps requests.

    answer returns an array of the reply and how many tool calls ran. Inside if (import.meta.main), answer What is the budget for WD1063? and print tools run: and the number, then the reply.

  3. Task 3

    budget.ts: a budget per conversation

    Write and export async prices(model), the model's price_inr_per_million from the API's list of models, and async cost(usage, model), a request's cost in rupees: cached prompt tokens (prompt_tokens_details, which may be missing) at cached_input, the other prompt tokens at input and completion tokens at output, per million. Write and export a class BudgetSpent extending Error, and a class Conversation, made as new Conversation(maxTokens, maxRupees, model = 'lab-chat'), with history, tokens and rupees spent, and async ask(message): before sending the history plus the message, throw a BudgetSpent if the tokens spent plus the request's tokens as baraat's tokens counts them pass maxTokens, or the rupees spent are maxRupees or more; else send it at temperature 0, keep the message and reply in the history, add usage total_tokens and the cost, and return the reply.

    Inside if (import.meta.main), send WD1063's thread messages as baraat's asMessage, one by one, to new Conversation(20000, 3); after each print turn, its number, a colon, the tokens spent, tokens, a comma and the rupees spent with ₹ and 4 decimals; when it stops print stopped before turn, the number, a colon, the tokens, tokens and, the rupees the same way and spent.

  4. Task 4

    referee.ts: a referee between two specialists

    Write and export async judge(question, a, b, reference, model = 'lab-chat'): ask at temperature 0, with an instruction to compare the answers against the reference and reply A, B or tie, and the lines Question:, Answer A:, Answer B: and Reference:, each with its text; return A or B when the reply is that letter (ignoring case, spaces and a full stop), else tie. Write and export async referee(question, first, second, reference, model = 'lab-chat'): judge with first as A, then with second as A; return the answer both verdicts pick, or null.

    Inside if (import.meta.main), ask What is the budget? of two budget specialists for WD1057 at temperature 0, one with notes holding its budget as booked, then one with the budget in memory, and print from the booking: and from memory: with their replies; then lab-chat picks: and lab-chat-biased picks: with the referee's choice (reference: budget: and the budget in memory), or a planner.

  5. Task 5 · interview question

    Interview: fix-me.ts, a loop detector that stops the wrong conversations

    firstRepeat(handoffs) in fix-me.ts should return the index of the first handover that repeats an earlier one, or null. Each handover is an object with from, to and reason; it repeats when an earlier one went from the same agent to the same agent for the same reason, the reasons compared without case and without spaces at the ends. Run it: a client who comes back to catering with a new question is stopped as a loop.

    Fix firstRepeat. Keep its name, its parameter and the export. The check tries real loops and conversations that only look like them.

What you will learn

  • Handoff loops between agents, and a hop limit (policy 6.4).
  • Tool loops: a model that repeats a call, and how to notice.
  • A budget per conversation, in tokens and in rupees, from the price list.
  • A referee: a judge that compares two answers against what memory knows.

More in Many agents

Build "When agents fail" yourself

Baraat Bliss, a Jaipur wedding planner, must remember months of family decisions across many agents. You build its assistant in TypeScript. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Memory and multi-agent systems in TypeScript are free. This one comes with the all labs plan.