AI engineering traps · Chapter 15 of 16

Sending the whole document every turn

Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Tools
  • 3 tasks
  • Python
  • In the all labs plan

The first 2 chapters of AI engineering traps are free. This one comes with the all labs plan.

What you build: 3 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    repro.py: ask eight questions in one chat, on purpose

    Write reproduce() in repro.py. Make one new BylawsChat from bylaws.py and ask it these eight questions, in this order, in the same chat: 'Can I keep a dog in my flat?', 'Until what time can we play loud music?', 'How much is the late fee on maintenance?', 'How many nights can a guest park?', 'When can I move furniture in?', 'Is the swimming pool open on Mondays?', 'Who may vote at the general body meeting?', 'Can I wash my car with a hose pipe?'. Stop at the first question that fails (catch the openai.APIStatusError) and return how many questions were answered.

    The check runs your reproduce() twice: beside the original bylaws.py it must return 6 (the seventh request no longer fits the model's window), and beside a fixed bylaws.py 8. BylawsChat uses the lab model when you leave out its client; reproduce() must finish within 10 seconds.

  2. Task 2

    bylaws.py: send the bylaws once, at the start

    Fix BylawsChat in bylaws.py. Put the bylaws in the system message once, after the instructions (SYSTEM, a blank line, 'Bylaws:', then BYLAWS), and send each question as a user message on its own. Then every request carries the bylaws exactly once, and every request starts the same way, so the prompt cache reuses that start.

    Keep ask, usage and the printed line bylaws: <n> prompt tokens, <m> cached as they are. The check asks a chat of nine questions of its own with the lab model: none may fail, each must still be answered from the right bylaw, every request must carry the bylaws exactly once in its first message, the system message, and from the second request on 1,024 or more prompt tokens must be cached. Sending the bylaws only with the newest question is not a fix. bylaws_text.py is not to be changed.

  3. Task 3

    test_bylaws.py: a test that would have caught it

    Write a pytest file, test_bylaws.py, that would have caught this before the bill. The check runs your file three times: beside the original bylaws.py, where at least one test must fail; beside the fixed bylaws.py, where every test must pass; and beside a wrong fix that sends the bylaws only with the newest question, where at least one test must fail.

    So ask one BylawsChat eight questions of your own, assert chat.usage has 8 entries (no question failed), and assert the second request's cached tokens, chat.usage[1].cached, are more than 0. Each test must finish within 10 seconds.

What you will learn

  • repro.py: ask eight questions in one chat, on purpose
  • bylaws.py: send the bylaws once, at the start
  • test_bylaws.py: a test that would have caught it

More in Tools

Build "Sending the whole document every turn" yourself

Sunwai, a Lucknow app for housing societies, runs on a model, and Farah, its CTO, keeps assigning you tickets. Each is a trap AI engineering interviews love to ask about. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of AI engineering traps are free. This one comes with the all labs plan.