APIs with Express · Chapter 19 of 29

Rate limiting

By the end of this chapter you can limit how often each caller uses your API, in three ways, and tell a caller who goes over when to come back. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Production
  • 5 tasks
  • Express
  • In the all labs plan

The first 2 chapters of APIs with Express are free. This one comes with the all labs plan.

Why you need this

On Friday at 4:02 pm the order API crawled: a bug in Bun Maska Co's new app asked for the menu 200 times a second, and every café waited behind it. Kabir wants a limit per partner, counted by its X-Api-Key, so one broken client cannot take everybody down.

A chapter 6 guard can refuse a request, but it remembers nothing between requests and knows nothing of time. A counter at the top of the file counts all cafés together, and one that never resets blocks a café for good.

What you build: 5 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    fixedWindow: a limit per API key per window

    Export fixedWindow({ limit, windowMs, now, key }), which returns a middleware. now is the clock (default Date.now) and key(req) names the caller (default: the X-Api-Key header, or the IP address without one). Time is cut into windows that start at multiples of windowMs: with windowMs 60000, a request at 10000 is in the window from 0 to 60000. Each key may make limit requests per window; another key has its own count.

    Every answer gets RateLimit-Limit (the limit), RateLimit-Remaining (what is left after this request: '2', '1', '0' for a limit of 3) and RateLimit-Reset (the seconds until the window ends, rounded up).

    Over the limit it answers 429 with { error: 'Too many requests', retryAfter } and a Retry-After header, both the seconds until the window ends, rounded up, and RateLimit-Remaining '0'. With limit 3 and windowMs 60000, the fourth request at now() 10000 gets retryAfter 50, and one at 59500 gets Retry-After '1'. At 60000 a new window starts: Remaining '2' and Reset '60'.

  2. Task 2

    slidingWindow: no burst at the window's edge

    Export slidingWindow({ limit, windowMs, now, key }), which returns a middleware: each key may make limit requests in any windowMs, wherever now() is. A request made at time t counts while now() is less than t + windowMs, and a refused request never counts.

    Every answer gets RateLimit-Limit and RateLimit-Remaining (what is left after this request). Over the limit it answers 429 with { error: 'Too many requests', retryAfter } and Retry-After, both the seconds until the oldest counted request stops counting, rounded up.

    With limit 3 and windowMs 60000: three requests at 59000, then one at 61000 gets 429 with retryAfter 58 (a fixed window would have reset at 60000). At 118999 it is still 429, and at 119000 the request goes through with Remaining '2'.

  3. Task 3

    tokenBucket: bursts allowed, refilled over time

    Export tokenBucket({ capacity, refillPerSecond, now, key }), which returns a middleware. Each key has a bucket that starts full with capacity tokens and refills all the time at refillPerSecond tokens a second (fractions count), never above capacity. A request takes one token when at least one whole token is there.

    Every answer gets RateLimit-Limit (the capacity) and RateLimit-Remaining (the whole tokens left after this request, rounded down). With no whole token it answers 429 with { error: 'Too many requests', retryAfter } and Retry-After, both the seconds until the next whole token, rounded up.

    With capacity 5 and 1 a second: five requests leave '4', '3', '2', '1', '0', and the sixth gets Retry-After '1'. 2.5 seconds later two more go through and the third does not. After an hour, only 5 go through. With capacity 1 and 0.25 a second, the second request gets retryAfter 4.

  4. Task 4

    createApp: limit /api per API key, never /health

    In createApp({ now }), everything under /api goes through your tokenBucket with capacity 5 and refillPerSecond 1, using the now that createApp was given, and keyed by the X-Api-Key header (the default key).

    GET /api/menu answers the menu. The sixth request with X-Api-Key kulhad answers 429 with Retry-After '1', while bunmaska still gets 200; a second later kulhad gets 200 again.

    GET /health answers { ok: true } every time, however often it is called, and has no RateLimit headers.

  5. Task 5 · interview question

    Interview: fixedWindow forgets keys from old windows

    The middleware that fixedWindow returns also has a size() method: the number of keys it is storing right now.

    Keys from older windows must not stay forever. When a request lands in a new window, every entry from an older window is deleted, before this request is counted. With limit 2 and windowMs 60000: twenty different keys at time 0 make size() 20, and one of them coming back keeps it at 20. At 60000 a new key makes size() 1, and visitor-0 coming back starts fresh with RateLimit-Remaining '1' and size() 2.

What you will learn

  • Counting per caller: a Map from each caller's key to its counters, kept by the middleware, with a clock passed in.
  • 429 and Retry-After: "too many, come back in this many seconds", and the RateLimit-* headers.
  • Three ways to count: fixed window, sliding window and token bucket, and what each gets wrong.
  • Memory: why a limiter must forget.

More in Production

Build "Rate limiting" yourself

Partner cafés want Tapri’s menu online by Friday: you build its API. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of APIs with Express are free. This one comes with the all labs plan.