AI engineering traps (TypeScript) · Chapter 11 of 16

Tool calls left unanswered

Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Tools
  • 3 tasks
  • TypeScript
  • In the all labs plan

The first 2 chapters of AI engineering traps (TypeScript) are free. This one comes with the all labs plan.

What you build: 3 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    repro.ts: ask two things at once, on purpose

    Write reproduce() in repro.ts. Build fake services, an object whose dues(flat) resolves to 4500 and nextTanker(society) to 'Saturday at 9 am'. Ask askSahayak from assistant.ts, for the resident { flat: 'B-304', society: 'Gomti Heights' }, the ticket's question exactly: 'What are my dues and when is the next water tanker?'. Resolve to the answer; if askSahayak throws, catch the error and resolve to 'error ' followed by its status (the error's status property).

    The check runs your reproduce() twice: beside the original assistant.ts it must resolve to 'error 400', and beside a fixed assistant.ts to 'Your dues are ₹4,500. The next water tanker comes on Saturday at 9 am.' askSahayak uses the lab model when you leave out its client; reproduce() must finish within 10 seconds.

  2. Task 2

    assistant.ts: answer every tool call the model made

    Fix askSahayak in assistant.ts. When the model asks for tools, run every call it made, in order, and send each result back as its own tool message whose tool_call_id is that call's id: one call, one tool message; two calls, two. Keep the assistant message with every call in it, the tools, runTool and the log line sahayak: the model asked for <n> tool calls as they are.

    The check uses its own resident and services with the lab model: a question that needs both tools must get the answer with both the dues and the tanker, after a request that answers each call by its id, and a question that needs one tool still works. It also type-checks assistant.ts. Dropping the second call from the assistant message is not a fix.

  3. Task 3

    assistant.test.ts: a test that would have caught it

    Write a vitest test file, assistant.test.ts, that would have caught this bug before residents met it. The check runs your file three times: beside the original assistant.ts, where at least one test must fail; beside the fixed assistant.ts, where every test must pass; and beside a wrong fix that drops the second tool call, where at least one test must fail.

    So ask a question that needs both tools, with fake services of your own, and expect the answer to contain both the dues and the tanker time. Each test must finish within 10 seconds.

What you will learn

  • repro.ts: ask two things at once, on purpose
  • assistant.ts: answer every tool call the model made
  • assistant.test.ts: a test that would have caught it

More in Tools

Build "Tool calls left unanswered" yourself

Sunwai, a Lucknow app for housing societies, runs on a model, and Farah, its CTO, keeps assigning you tickets. Each is a trap AI engineering interviews love to ask about. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of AI engineering traps (TypeScript) are free. This one comes with the all labs plan.