AI engineering traps (TypeScript) · Chapter 1 of 16
A reply cut short
Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Calling the model
- 3 tasks
- TypeScript
- Free chapter
Free with a Google or GitHub sign-in, like the first 2 chapters of every lab.
Why you need this
A chat model writes its reply one token at a time (a token is a word or a piece of one). It stops for one of a few reasons, and every reply says which one in finish_reason: "stop" when it finished on its own, "length" when it hit the max_tokens you allowed, and "tool_calls" when it wants a tool (case 10 onwards).
Here Sahayak, Sunwai's assistant for residents, drafts a reminder with max_tokens: 20:
import OpenAI from 'openai';
const client = new OpenAI();
const res = await client.chat.completions.create({
model: 'lab-chat',
max_tokens: 20,
messages: [
{ role: 'system', content: "You are Sahayak, Sunwai's assistant for residents." },
{ role: 'user', content: 'Draft a reminder about the Holi party on the terrace' },
],
});
console.log(res.choices[0].finish_reason, res.usage?.completion_tokens);
console.log(res.choices[0].message.content);
length 20
Here is a draft you can send:
Dear residents,
This is a note about the Holi party
What the fix in this case does with the ticket's notice, step by step:
The lesson
By the end of this case
You can tell a reply that was cut off from its symptoms, make it happen on purpose with the lab model, fix the code so it reads finish_reason before it trusts the text, and pin it down with a test that fails on the bug and passes on the fix.
The ticket
SUN-212: The water notice to Gomti Heights ended mid-sentence. Assigned to you by Farah. Priority: high. Since: last night.
Rohit, in the operations channel at 7:40 am: "Gomti Heights sent their water notice through the app last night. It went to 400 families and it just stops: Lift B will be shut from 11 am. Shut till when? Mrs. Rizvi is getting calls, and two people with old parents were stuck downstairs this morning. The notice also has no sign-off, so people think it is fake."
The notice writer's log, from last night:
21:14:07.311 notices notice: 214 characters
21:14:07.402 notices sent to 400 residents of Gomti Heights
Farah: "Nothing failed. The API said 200, the code got a string, the string went out. Short notices have been fine for months, and this was the longest one a committee has ever written. Reproduce it first, then fix it, then make sure it can never come back. You are in notice.ts."
Why nothing complained
Look at what did not happen. No error, no exception, a status of 200. A cut reply is a normal, successful reply: you asked for at most 20 tokens and got exactly that. The text is a plain string, and typecheck prints No type errors. The only sign is one field most code never reads.
max_tokens is a limit you set to keep replies short and cheap. That is a good idea, as long as the code checks whether the limit was hit. When it was, the honest choices are to ask again with more room, or to fail loudly. Sending half a notice is not one of them.
Making it happen on purpose
The lab model answers the same way every time, so the ticket's notice is cut in the same place on every run. That is your repro: the real code, the real facts, the real cut.
A test that would have caught it
The lab has vitest: it names one test, expect checks a value. A test can also hand the code a fake model: an object with the same shape as the client, answering whatever you decide. This one is always cut off:
const cutOff = {
chat: { completions: { create: async () => ({
choices: [{ finish_reason: 'length', message: { content: 'Welcome to Gomti' } }],
}) } },
} as unknown as OpenAI;
it('refuses a welcome that was cut off', async () => {
await expect(welcome('Zehra', cutOff)).rejects.toThrow();
});
Beside a welcome that never reads finish_reason it fails (trimmed):
× refuses a welcome that was cut off 5ms
AssertionError: promise resolved "'Welcome to Gomti'" instead of rejecting
A good guard fails on the bug and passes on the fix, so it checks what must be true of a good notice: every fact is there, and it ends with the sign-off.
Your turn
Three tasks, one file each. In repro.ts, cut the ticket's notice short on purpose, and run it with node repro.ts in the case's folder. In notice.ts, read finish_reason and never return a cut notice; typecheck shows any type errors. In notice.test.ts, write the guard: the check runs it beside the original code, the fix and a wrong fix that trims a cut notice to its last full stop. Then answer Farah's interview question at the bottom of the case, in your own words.
What you build: 3 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
repro.ts: cut the water notice short on purpose
Write reproduce() in repro.ts. Ask writeNotice from notice.ts for the ticket's notice, with these four lines as the facts, word for word: Society: Gomti Heights, then - The water supply will be off on Saturday 8 November from 10 am to 4 pm while the overhead tank is cleaned., then - Please store enough water on Friday night., then - Lift B will be shut from 11 am to 1 pm for its monthly service. Resolve to the last line of the notice it gives back.
The check runs your reproduce() twice: beside the original notice.ts it must resolve to 'Lift B will be shut from 11 am' (the notice in the ticket), and beside a fixed notice.ts to 'Managing Committee, Gomti Heights'. So it must really ask for the ticket's notice. writeNotice uses the lab model when you leave out its client; reproduce() must finish within 10 seconds.
- Task 2
notice.ts: never send a notice that was cut off
Fix writeNotice in notice.ts. Keep the first ask as it is, with max_tokens 50, and then read finish_reason: 'stop' means the notice is complete, so return it; 'length' means it hit max_tokens and was cut off, so ask once more with the same messages and max_tokens 400, and return that notice if it is complete. If that one is cut off too, throw an Error saying the notice was cut off.
Keep the log line notice: <n> characters for the notice you return, and keep writeNotice(facts, client?) as it is. The check uses its own facts with the lab model (a short notice must take one ask, a long one two), and fake models that are cut off twice. It also type-checks notice.ts. Trimming a cut notice back to its last full stop is not a fix.
- Task 3
notice.test.ts: a test that would have caught it
Write a vitest test file, notice.test.ts, that would have caught this bug before the notice reached 400 families. The check runs your file three times: beside the original notice.ts, where at least one test must fail; beside the fixed notice.ts, where every test must pass; and beside a wrong fix that trims a cut notice to its last full stop, where at least one test must fail.
So ask for a notice too long for 50 tokens, with facts of your own, and check that every fact is in it and that its last line is the committee's sign-off. Each test must finish within 10 seconds.
What you will learn
- repro.ts: cut the water notice short on purpose
- notice.ts: never send a notice that was cut off
- notice.test.ts: a test that would have caught it
More in Calling the model
Build "A reply cut short" yourself
Sunwai, a Lucknow app for housing societies, runs on a model, and Farah, its CTO, keeps assigning you tickets. Each is a trap AI engineering interviews love to ask about. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
Free with a Google or GitHub sign-in, like the first 2 chapters of every lab.