Concurrency and races · Chapter 10 of 15
The retry storm
Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Time
- 3 tasks
- JavaScript
- In the all labs plan
The first 2 chapters of Concurrency and races are free. This one comes with the all labs plan.
What you build: 3 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
repro.js: hammer a provider that is down, on purpose
Write reproduce() in repro.js, on a pretend clock. Keep a pretend time, now, starting at 0. Make a fake sleep(ms) that only moves now on by ms, and a fake send() that throws an Error while now is under 1000 (the SMS provider is down for its first second), counting those calls, and resolves to 'sent' once now is 1000 or more. Call callWithRetry(send, { sleep, random }) from retry.js with a random that always answers 1, catch the rejection if there is one, and resolve to how many calls reached the provider while it was down.
The check runs your reproduce() twice: beside the original retry.js it must resolve to 6 (six calls at time 0, then it gives up), and beside a fixed retry.js it must resolve to 4 (calls at 0, 100, 300 and 700 ms, then 'sent' at 1500). No real waiting: your sleep resolves at once, and reproduce() must finish within 2 seconds.
- Task 2
retry.js: back off, with jitter
Fix callWithRetry(fn, { tries, base, cap, sleep, random }) in retry.js. After a call fails, if more tries are left, wait with await sleep(ms) before calling again, where ms for retry n (n is 0 before the second call, 1 before the third, and so on) is Math.floor(random() * Math.min(cap, base * 2 ** n)): exponential backoff with full jitter. When a call works, resolve to its value and call no more. When all tries calls failed, reject with the last error, with no wait after the last call. A call that never fails is called once, with no wait.
Keep the log line retry: call <k> of <tries> failed (<message>), the defaults (tries 6, base 100, cap 5000, a real sleep and Math.random) and the export as they are. The check hands you its own sleep and random, writes down every wait, checks the cap, and holds a sleep back to see that the next call waits for it.
- Task 3
retry.test.js: a test that would have caught it
Write a vitest test file, retry.test.js, that would have caught the storm before the release. The check runs your file three times: beside the original retry.js, where at least one test must fail; beside the fixed retry.js, where every test must pass; and beside a wrong fix that backs off but ignores random (no jitter), where at least one test must fail.
So call callWithRetry with an fn that keeps failing, a fake sleep that writes down every wait and a random that answers a number other than 1, and expect the exact waits (for base 100 and random 0.5: 50, 100, 200, 400 and 800), and that it rejects with the last error. No real waiting: each test must finish within 3 seconds.
What you will learn
- repro.js: hammer a provider that is down, on purpose
- retry.js: back off, with jitter
- retry.test.js: a test that would have caught it
More in Time
Build The retry storm yourself
Tapri's backend grew busy, and Kabir keeps handing you incidents that only happen when two things run at once. Each is a race interviews love to ask about. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of Concurrency and races are free. This one comes with the all labs plan.