AI engineering traps · Chapter 8 of 16
A cache on the wrong key
Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Calling the model
- 3 tasks
- Python
- In the all labs plan
The first 2 chapters of AI engineering traps are free. This one comes with the all labs plan.
What you build: 3 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
repro.py: give Aliganj Greens the wrong rule on purpose
Write reproduce() in repro.py. With one FaqBot from faq.py, ask in English, word for word, How much is guest parking? first for 'Gomti Heights', then for 'Aliganj Greens'. Return the answer Aliganj Greens gets.
The check runs your reproduce() twice: beside the original faq.py it must return 'Guest parking costs ₹50 a night, paid at the gate.' (Gomti Heights' rule), and beside a fixed faq.py 'Guest parking is free until 10 pm.'. So both asks must go through the same FaqBot. FaqBot uses the lab model when you leave out its client; reproduce() must finish within 10 seconds.
- Task 2
faq.py: key the cache on everything that is sent
Fix ask in faq.py so the cache key holds everything that goes into the request: the model, the temperature and every message (the system message names the society and the language, and holds the rules). Two asks share a cached answer only when they would send the very same request, and then the second ask must not call the model again.
Keep RULES, request_for and the printed lines faq: cache hit for <society> and faq: asked the model for <society>. The check asks in both societies and both languages and counts the model calls. A key of the society and the question is not a fix.
- Task 3
test_faq.py: a test that would have caught it
Write a pytest file, test_faq.py, that would have caught this bug. The check runs your file three times: beside the original faq.py, where at least one test must fail; beside the fixed faq.py, where every test must pass; and beside a wrong fix that keys on the society and the question, where at least one test must fail.
So, with one FaqBot and the lab model, ask the same question for two societies, and in two languages, and assert each answer is that society's rule in that language (RULES holds them). Each test must finish within 10 seconds.
What you will learn
- repro.py: give Aliganj Greens the wrong rule on purpose
- faq.py: key the cache on everything that is sent
- test_faq.py: a test that would have caught it
More in Calling the model
Build "A cache on the wrong key" yourself
Sunwai, a Lucknow app for housing societies, runs on a model, and Farah, its CTO, keeps assigning you tickets. Each is a trap AI engineering interviews love to ask about. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of AI engineering traps are free. This one comes with the all labs plan.