LangChain and LangGraph · Chapter 22 of 25

Errors, retries and limits

By the end of this chapter you can say what goes wrong when a program talks to a model over the network, let a request try again with with_retry, hand it to a backup model with with_fallbacks, put a time limit on a slow model, stop an agent that loops with recursion_limit, and write a node that fails without taking the whole graph down. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Production
  • 4 tasks
  • Python
  • In the all labs plan

The first 2 chapters of LangChain and LangGraph are free. This one comes with the all labs plan.

Why you need this

Saturday night, 11 pm. The provider's servers are busy and RoomRaja's assistant crashes on every third message. Mrunal wakes up to forty unanswered tenants. On Monday Faisal finds one conversation that cost more than a whole day of normal traffic: the agent had asked for the same tool again and again until the conversation no longer fit in the model.

Nothing in your code was wrong. A model is a server on the other side of a network, and networks and servers fail. Everything you built so far assumes every invoke comes back with an answer. This chapter is about the times it does not, and what LangChain and LangGraph give you for them.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    retries.py: try again

    Write ask(llm, question): wrap the chat model it is given with with_retry so a failed request is tried again, up to 3 attempts in all, and return the text of the reply that finally comes. After the third failure, let the error out of ask.

    The check gives ask a ChatOpenAI for lab-chat-flaky with max_retries=0 (it fails with a 503, then a 429, then answers) and must get the answer, then one for lab-chat-down (always 503) and must see exactly 3 requests and an error.

  2. Task 2

    fallback.py: a backup model

    lab-chat-down is RoomRaja's main model tonight, and it is down. Make a ChatOpenAI for lab-chat-down with its own retries turned off (max_retries=0), give it lab-chat-mini as a fallback with with_fallbacks, and ask the tenant's question exactly:

    Can I shift to a single room next month?

    Print one line: the name of the model that answered, read from the reply's response_metadata, a colon and a space, then the reply's text. lab-chat-down must be asked once only.

  3. Task 3

    loop_limit.py: a limit that answers politely

    Write answer(question, model="lab-chat"): make an agent with create_agent, a ChatOpenAI for that model, one tool ticket_status (it returns roomraja.ticket_status for a ticket id), and the system prompt You are RoomRaja's tenant assistant. Use the tools. Run the question through it with a recursion_limit of 6 and return the text of its last message.

    When the limit stops the run, return exactly: Sorry, I could not finish that. Mrunal will reply to you soon.

    The check asks about TK5012 with lab-chat, which answers, and with lab-chat-loopy, which asks for the same tool again forever.

  4. Task 4 · interview question

    Interview: resilient.py, retry or fall back

    Write resilient(primary, backup), returning one model (a Runnable) that tries primary up to 3 times, but only for errors that can pass with time: a rate limit (429), a server error (5xx), a timeout or a lost connection. When primary still fails, or fails with any other error, it asks backup.

    The check makes the models itself with max_retries=0 and runs four cases: lab-chat-flaky first; lab-chat-down first; a message too long for lab-chat (400); and a model that does not exist (404). The interviewer will ask: which failures do you retry, which do you fall back from, and why does the order of the two matter?

What you will learn

  • What fails, and which failures pass if you wait a second.
  • with_retry: try the same request again, a few times.
  • Timeouts and with_fallbacks: stop waiting, and ask another model.
  • recursion_limit: a ceiling on how many steps a graph may take.
  • A node that fails safely: catch the error inside the graph.

More in Production

Build "Errors, retries and limits" yourself

RoomRaja, a Pune co-living company, answers tenants on WhatsApp by hand. You build its assistant. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of LangChain and LangGraph are free. This one comes with the all labs plan.