Building with LLMs · Chapter 24 of 24
Interview round
By the end of this chapter you have answered five timed questions, one idea each, from across the whole lab: counting a request's tokens, retrying a failing API, asking many questions at once without breaking a limit, reading a sloppy model's JSON, and finding the nearest FAQ by meaning. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Building with LLMs
- 5 tasks
- Python
- In the all labs plan
The first 2 chapters of Building with LLMs are free. This one comes with the all labs plan.
Why you need this
Nikhil gives every new AI engineer the same round before they own a feature: "Short problems, ten minutes each. I am not checking that you remember the API. I am checking that you know what happens on the wire, and that you measure instead of guessing." Zoya wants the next hire interviewed the same way, so the round is now part of the lab.
Each question uses one idea from an earlier chapter, with no hand-holding: the file has the goal, the check has the edge cases. The ideas are below, as a refresher and as the things an interviewer listens for. The answers are not.
What you build: 5 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1 · interview question
Interview: fits.py, will it fit?
Time yourself: 10 minutes. Write count_tokens(messages), the number of prompt tokens lab-chat counts for a list of messages with text content, exactly as usage.prompt_tokens reports it, and fits(messages, max_tokens), True when the prompt and a reply of max_tokens fit together in lab-chat's 8,192-token window.
The check compares count_tokens with lab-chat on English, Hindi and emoji messages, and tries fits at the exact edge of the window.
- Task 2 · interview question
Interview: retry.py, retry like a professional
Time yourself: 10 minutes. Write ask_with_retries(messages, model="lab-chat"), which returns the reply's text. A 429, a server error (500 or above) or a connection error is retried up to 3 times after the first try, waiting before each retry as many seconds as the retry-after header says, or a short growing time without one; after the last, raise that error. Any other error is raised at once.
The check counts your requests to lab-chat-flaky (503, then 429 with retry-after: 1, then an answer), lab-chat-down (always 503) and a model that does not exist, and times the wait after the 429.
- Task 3 · interview question
Interview: ask_all.py, many questions, a few at a time
Time yourself: 10 minutes. Write the async function ask_all(questions, model="lab-chat", limit=3): ask model each question as one user message, all at the same time but never more than limit waiting at once, and return the replies' texts in the order of the questions.
The check asks lab-chat-slow (4 seconds per answer) 6 questions with limit 3 and times it: about 8 seconds is right, 24 means one at a time, 4 means no limit.
- Task 4 · interview question
Interview: repair_json.py, read sloppy JSON
Time yourself: 10 minutes. Write parse_json(text): remove a markdown code fence around the reply (three backticks, maybe followed by json, at the start, and three at the end) if there is one, remove every comma just before a closing } or ], and parse the JSON; text that is still not JSON raises a ValueError. A comma inside a value, like "Ravi, Pune", stays.
Then extract(message): ask lab-chat-sloppy at temperature 0 with SYSTEM (given in the file) and the message, and return parse_json of the reply.
- Task 5 · interview question
Interview: nearest_faq.py, the closest FAQ question
Time yourself: 10 minutes. Write nearest(question): the help-centre question from /workspace/data/faq.md (the lines in bold, without the **) whose lab-embed embedding has the highest cosine similarity with the question's.
Embed the FAQ's questions in one request. The check tries English and Hinglish questions and compares your answer with its own.
What you will learn
- How to work a timed question: say the plan, build the smallest thing that runs, then measure it against the real API.
- The five ideas again, at interview depth: tokens (chapter 3), retries (6), concurrency (7), JSON from a model (11 and 12), and embeddings (18).
More in Building with LLMs
Build "Interview round" yourself
SafarLine, an intercity bus company, drowns in WhatsApp messages. You build its first AI assistant. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of Building with LLMs are free. This one comes with the all labs plan.