The Vercel AI SDK · Chapter 18 of 21
Errors, retries and abort
By the end of this chapter you can tell a provider's short hiccup from a real outage, decide how long Mali may take, say sorry kindly when it fails, let the customer try again, and stop the model when the customer stops reading. You will read the SDK's errors, set retries and timeouts, give the chat route a safe error message and a stop signal, and add Retry to the chat. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Ready for customers
- 4 tasks
- TypeScript
- In the all labs plan
The first 2 chapters of The Vercel AI SDK are free. This one comes with the all labs plan.
Why you need this
Chapter 17 gave Mali a backup model. Vikram is glad, but a backup is the last line, not the first: "Most failures are not outages. A provider is busy for a second, or slow for ten. What does Mali do then? And when a customer gives up and closes the chat, are we still paying for the answer?"
Deepa has a different worry. She tried Mali on a bad afternoon and saw this under her question: "An error occurred." No idea what happened, no way to try again, so she typed the whole question a second time. "If I do that, customers will too. Then they call us."
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
errors.ts: a hiccup or an outage?
Export async function ask(modelId, question, maxRetries?): it asks lab(modelId) the question with MALI_INSTRUCTIONS, passes maxRetries to generateText, and returns result.text. When it fails, it returns one line instead:
for a RetryError (RetryError.isInstance): RetryError after N attempts, last status S, with N = error.errors.length and S = error.lastError's statusCode;
for an APICallError (APICallError.isInstance): APICallError S, retryable: true or false, from error.statusCode and error.isRetryable;
anything else is thrown again.
- Task 2
timeouts.ts: give up in time
Export TOO_SLOW, a kind sentence for a customer, and async function askInTime(question, ms): it asks lab('lab-chat-slow') (4 s to answer) with MALI_INSTRUCTIONS and timeout: ms, and returns result.text, or TOO_SLOW when the error thrown is named TimeoutError; any other error is thrown again.
The Check asks with 1000 ms (TOO_SLOW in about 1 s, and the request closed, status 499) and with 8000 ms (the answer).
- Task 3
server.ts: fail kindly, stop in time
Export SORRY, a kind sentence, and chatRoute(model): a function that returns a chat route, POST(req), as in chapter 9 (streamText with the given model, MALI_INSTRUCTIONS, the converted UI messages, maliTools() and isStepCount(5), returned with createUIMessageStreamResponse and toUIMessageStream), plus abortSignal: req.signal and an onError that logs on streamText, and onError: () => SORRY on toUIMessageStream. Then export const POST = chatRoute(lab('lab-chat')).
The Check posts to chatRoute(lab('lab-chat-down')) and wants one error event that says SORRY; and it stops reading a long answer after 5 pieces and wants the model's request closed (499).
- Task 4
Chat.tsx: an error the customer can retry
In the given chat, take error and regenerate from useChat. While error is set, show error.message and a button named Retry that calls regenerate().
The Check's route fails the first time (its stream ends with an error event) and works after: the page should show the error's message, then, after Retry, the answer, the error gone, and the customer's question only once.
What you will learn
- Retries: what the SDK does by itself when a call fails, and maxRetries.
- The SDK's errors: APICallError and RetryError, checked with isInstance.
- Timeouts: timeout and abortSignal, and what a time limit saves.
- Errors in a stream: onError for your log, onError for the customer.
- Retry in the UI: useChat's error and regenerate.
- Stop that reaches the model: abortSignal: req.signal.
More in Ready for customers
Build "Errors, retries and abort" yourself
HariBhari, a Bengaluru plant shop, wants Mali, a plant helper on its website. You build it end to end, from the first call to a chat that streams cards, asks before it adds to the cart and stays within budget. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of The Vercel AI SDK are free. This one comes with the all labs plan.