Building with LLMs · Chapter 24 of 24

Interview round

By the end of this chapter you have answered five timed questions, one idea each, from across the whole lab: counting a request's tokens, retrying a failing API, asking many questions at once without breaking a limit, reading a sloppy model's JSON, and finding the nearest FAQ by meaning. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Building with LLMs
  • 5 tasks
  • Python
  • In the all labs plan

The first 2 chapters of Building with LLMs are free. This one comes with the all labs plan.

Why you need this

Nikhil gives every new AI engineer the same round before they own a feature: "Short problems, ten minutes each. I am not checking that you remember the API. I am checking that you know what happens on the wire, and that you measure instead of guessing." Zoya wants the next hire interviewed the same way, so the round is now part of the lab.

Each question uses one idea from an earlier chapter, with no hand-holding: the file has the goal, the check has the edge cases. The ideas are below, as a refresher and as the things an interviewer listens for. The answers are not.

What you build: 5 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1 · interview question

    Interview: fits.py, will it fit?

    Time yourself: 10 minutes. Write count_tokens(messages), the number of prompt tokens lab-chat counts for a list of messages with text content, exactly as usage.prompt_tokens reports it, and fits(messages, max_tokens), True when the prompt and a reply of max_tokens fit together in lab-chat's 8,192-token window.

    The check compares count_tokens with lab-chat on English, Hindi and emoji messages, and tries fits at the exact edge of the window.

  2. Task 2 · interview question

    Interview: retry.py, retry like a professional

    Time yourself: 10 minutes. Write ask_with_retries(messages, model="lab-chat"), which returns the reply's text. A 429, a server error (500 or above) or a connection error is retried up to 3 times after the first try, waiting before each retry as many seconds as the retry-after header says, or a short growing time without one; after the last, raise that error. Any other error is raised at once.

    The check counts your requests to lab-chat-flaky (503, then 429 with retry-after: 1, then an answer), lab-chat-down (always 503) and a model that does not exist, and times the wait after the 429.

  3. Task 3 · interview question

    Interview: ask_all.py, many questions, a few at a time

    Time yourself: 10 minutes. Write the async function ask_all(questions, model="lab-chat", limit=3): ask model each question as one user message, all at the same time but never more than limit waiting at once, and return the replies' texts in the order of the questions.

    The check asks lab-chat-slow (4 seconds per answer) 6 questions with limit 3 and times it: about 8 seconds is right, 24 means one at a time, 4 means no limit.

  4. Task 4 · interview question

    Interview: repair_json.py, read sloppy JSON

    Time yourself: 10 minutes. Write parse_json(text): remove a markdown code fence around the reply (three backticks, maybe followed by json, at the start, and three at the end) if there is one, remove every comma just before a closing } or ], and parse the JSON; text that is still not JSON raises a ValueError. A comma inside a value, like "Ravi, Pune", stays.

    Then extract(message): ask lab-chat-sloppy at temperature 0 with SYSTEM (given in the file) and the message, and return parse_json of the reply.

  5. Task 5 · interview question

    Interview: nearest_faq.py, the closest FAQ question

    Time yourself: 10 minutes. Write nearest(question): the help-centre question from /workspace/data/faq.md (the lines in bold, without the **) whose lab-embed embedding has the highest cosine similarity with the question's.

    Embed the FAQ's questions in one request. The check tries English and Hinglish questions and compares your answer with its own.

What you will learn

  • How to work a timed question: say the plan, build the smallest thing that runs, then measure it against the real API.
  • The five ideas again, at interview depth: tokens (chapter 3), retries (6), concurrency (7), JSON from a model (11 and 12), and embeddings (18).

More in Building with LLMs

Build "Interview round" yourself

SafarLine, an intercity bus company, drowns in WhatsApp messages. You build its first AI assistant. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Building with LLMs are free. This one comes with the all labs plan.