Build an AI gateway (TypeScript) · Chapter 21 of 21
Interview round
By the end of this chapter you have answered five questions an interviewer for a platform or AI infrastructure role asks about gateways, each one the way this lab taught you: say what is wrong in one sentence, fix it, and prove the fix with a small measurement. A retry loop that storms, a cache key that mixes teams, a breaker that never closes, a log line with a secret, and a p95 computed wrong. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Run it in production
- 5 tasks
- TypeScript
- In the all labs plan
The first 2 chapters of Build an AI gateway (TypeScript) are free. This one comes with the all labs plan.
Why you need this
Darwaza runs Jawab's day. Jawab is hiring a second platform engineer, and Farhan runs the interview the way he built Darwaza with you: "I will not ask what a circuit breaker is. I will show you code that ran in production and hurt someone, and ask what is wrong. Then you fix it, and show me a number that says it is fixed. Ten minutes each."
Each question is real code with a real bug, given in the file. The bugs are the ones gateways actually ship with: each looks reasonable, passes a quick test, and fails on a bad day. The ideas behind them are below, as a refresher and as what an interviewer listens for. The fixes are yours.
What you build: 5 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1 · interview question
Interview: storm.ts, a retry loop that storms
Time yourself: 10 minutes. The given withRetries retries forever, every 100 ms. Fix it, keeping its name and parameters (target, req, deadlineMs = 3000): at most 3 attempts in all, only for retryable errors; before each retry wait waitFor(error, attempt) from ./darwaza.ts (the provider's retry-after, or full jitter); and when the wait alone would end after the deadline, return the error at once.
The check makes Kestrel answer 529 to 30 requests at once and times when their retries leave, sends a 429 with a 300 ms retry-after, and a 429 whose 2,000 ms retry-after is past a 1,500 ms deadline.
- Task 2 · interview question
Interview: cache-key.ts, a cache key that mixes teams
Time yourself: 10 minutes. The given cacheKey is the feature and the last message. Fix it: a sha256 of everything that changes the answer, in a fixed order: the team, the feature, the model asked for, every message (role and content, the system prompt too), maxTokens and temperature; not the request's id.
The given main block asks kaapi's and nila's "Do you ship outside India?" through the given cached gateway; fixed, nila gets its own answer. The check also tries every field one at a time.
- Task 3 · interview question
Interview: breaker.ts, a breaker that never closes
Time yourself: 10 minutes. Run node breaker.ts: the given Breaker opens after a good probe. Fix it, keeping its name, constructor (cooldownMs, now) and methods: after the cooldown, one probe at a time (half-open); a probe that answers closes it and the failures that opened it no longer count; a probe that fails opens it again with a new cooldown; once closed, it opens only after minCalls recent calls with half of them failed.
- Task 4 · interview question
Interview: log-line.ts, a log line with a secret
Time yourself: 10 minutes. The given logAttempt logs the URL and the headers. Fix it, keeping its name and parameters (target, url, headers, status, ms): one JSON line with target, path (the URL's path, without the query string), status and ms; never the headers; and none of Darwaza's keys anywhere in the line.
The check logs a call to each provider as build() makes it, a Tara URL with ?key=, and a key it has never seen.
- Task 5 · interview question
Interview: p95.ts, a p95 computed wrong
Time yourself: 10 minutes. The given oldP95 is the old dashboard's. Write and export p95(results): the p95 of the ms of the results not refused, sorted as numbers, by nearest rank (position ceil(0.95 x count), counting from 1), 0 when there are none; not with percentile().
Under if (import.meta.main), replay 10:00 to 11:00 through v0 and print old p95: N ms (oldP95 of the results) then p95: N ms (yours).
What you will learn
- How to answer a gateway question: the cause in one sentence, the smallest fix, and a measurement.
- Five ideas again, at interview depth: retry storms (chapter 7), cache keys (11), the breaker's way back (9), secrets in logs (17) and percentiles (1, 17).
More in Run it in production
Build "Interview round" yourself
Jawab, a Kochi support desk, answers customers for six Kerala brands on three AI providers that fail, slow down and cost different amounts. You build Darwaza, its gateway, and replay a whole day of traffic against an SLO and a budget. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of Build an AI gateway (TypeScript) are free. This one comes with the all labs plan.