AI engineering traps (TypeScript) · Chapter 13 of 16
Retries that hammer
Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Tools
- 3 tasks
- TypeScript
- In the all labs plan
The first 2 chapters of AI engineering traps (TypeScript) are free. This one comes with the all labs plan.
What you build: 3 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
repro.ts: send a prompt that is too long, on purpose
Write reproduce() in repro.ts. Make an OpenAI client that counts its requests, by handing it its own fetch (new OpenAI({ maxRetries: 0, fetch: ... }) that adds one to a counter and then calls the real fetch). Send through askWithRetry from retry.ts, with that client, a request too long for the window: model 'lab-chat' and one user message whose content is 'water ' repeated 9000 times. It fails: catch the error, and resolve to how many requests the client sent.
The check runs your reproduce() twice: beside the original retry.ts it must resolve to 5, and beside a fixed retry.ts to 1 (a 400 is never retried). So count real requests. reproduce() must finish within 10 seconds.
- Task 2
retry.ts: retry only what can succeed, and wait first
Fix askWithRetry in retry.ts. Make at most 4 attempts in all. After a 429, any 5xx status, or a request that never got an answer (OpenAI.APIConnectionError), wait with sleep(ms) and ask again; any other error (400, 401, ...) is thrown at once, with no retry; when the 4th attempt fails too, throw its error. Wait what the error's retry-after header says, in seconds turned into milliseconds; with no such header, wait 500 ms, then 1000, then 2000.
Keep the log line retry: attempt <n> failed for every failed attempt, and keep askWithRetry(body, client?, sleep?) as it is. The check counts the requests and every sleep, with lab-chat-flaky (a 503, then a 429 with retry-after 1, then an answer), lab-chat-down (always 503), a prompt too long for the window, a request with no API key and a connection that fails once. It also type-checks retry.ts. Waiting before every retry is not enough if a 400 is still retried.
- Task 3
retry.test.ts: a test that would have caught it
Write a vitest test file, retry.test.ts, that would have caught this retry storm. The check runs your file three times: beside the original retry.ts, where at least one test must fail; beside the fixed retry.ts, where every test must pass; and beside a wrong fix that waits with backoff but still retries a 400, where at least one test must fail.
So test both sides, with a counting client and a fake sleep that only notes the milliseconds: a prompt too long for the window makes 1 request and no waits; 'lab-chat-flaky' (with something unique in the message) makes 3 requests with waits of 500 and 1000. Each test must finish within 10 seconds.
What you will learn
- repro.ts: send a prompt that is too long, on purpose
- retry.ts: retry only what can succeed, and wait first
- retry.test.ts: a test that would have caught it
More in Tools
Build "Retries that hammer" yourself
Sunwai, a Lucknow app for housing societies, runs on a model, and Farah, its CTO, keeps assigning you tickets. Each is a trap AI engineering interviews love to ask about. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of AI engineering traps (TypeScript) are free. This one comes with the all labs plan.