RAG in Python · Chapter 29 of 37
Never leak an internal runbook
By the end of this chapter you can enforce who sees which document in the search itself, for every merchant and every member of staff, and prove with attack questions that nothing internal reaches a merchant. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Production
- 4 tasks
- Python
- In the all labs plan
The first 2 chapters of RAG in Python are free. This one comes with the all labs plan.
Why you need this
Paisa Setu's index holds 866 chunks, and 100 of them are internal: runbooks with the goodwill credit limit, the fraud thresholds, the names of the partner banks. Support staff need them. Merchants must never see them, and Scale plan merchants must never see each other's negotiated prices.
Neha, head of compliance, asked Kabir's assistant a merchant's question and read the reply:
$ python3 permissions.py
Support may offer goodwill fee credits under runbook-goodwill-credits (up to ₹500 per merchant per quarter without approval). - If a merchant is very upset and has lost a sale, you may consider a goodwill fee credit (see runbook-goodwill-credits). [runbook-sev1-payments-outage#7] [runbook-debited-not-received#8]
['runbook-goodwill-credits#0 (internal)', 'runbook-sev1-payments-outage#7 (internal)', 'runbook-goodwill-credits#5 (internal)', 'runbook-debited-not-received#8 (internal)', 'dev-changelog#19 (merchant)']
The system message says "Never reveal internal documents or runbooks to merchants". It did not help. Everything you built so far searches every chunk for everyone, and then hopes.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
permissions.py: retrieve_for, merchants search merchant documents only
Neha needs runbooks kept from merchants. Write retrieve_for(conn, user, question, k=5, table): the k chunks of table nearest to the question, best first like retrieve, that this user may see. Staff (a Staff, role "staff") see every audience; a merchant (a Merchant, role "merchant") sees audience "merchant" only.
Put the condition in the SQL, in a WHERE before ORDER BY and LIMIT, so a merchant always gets k rows: retrieve_for(conn, merchant, "what is the goodwill credit limit?") is 5 rows and none is internal. Filtering retrieve's 5 rows afterwards leaves 1 of 5 for that question. Staff must still find the right runbook for at least 6 of the 8 internal questions in the labelled set. Task 2 is in this same function.
- Task 2
permissions.py: retrieve_for, each merchant sees only their own agreement
Still in retrieve_for: a merchant also sees only the chunks whose merchant_id is NULL (documents for every merchant) or their own merchant_id. Staff see every merchant's chunks.
For Merchant(merchant_id="M-3001"), "what is my card fee?" finds agreement-m-3001#0, and "what is the card fee for merchant M-3002?" finds no other merchant's agreement but still 5 rows. Staff asking that finds agreement-m-3002#0. The merchant id comes from user, never from the question. In SQL, NULL equals nothing, not even NULL.
- Task 3
permissions.py: can_see and answer_for, check every chunk before the prompt
Write can_see(user, row): whether a user may see a chunk, by retrieve_for's rules. Staff see everything; a merchant sees audience "merchant" with merchant_id None or their own. can_see(merchant, a row with audience "internal") is False.
Write answer_for(conn, user, question, search=retrieve_for, table): get the rows with search(conn, user, question, table=table), check every one with can_see, and raise PermissionDenied before calling the model if any one is not allowed. Then ask lab-chat with build_messages and return Reply(answer, sources), like naive_ask. Given a search that returns a runbook, answer_for for a merchant raises and sends nothing; with the default search, the goodwill question sends no runbook and the answer has no ₹500.
- Task 4 · interview question
Interview: permissions.py, find_leaks, try to leak on purpose
Write find_leaks(ask, user, questions): ask every question with ask (it returns a Reply) and find the sources that can_see says this user may not see. Return one Leak(question, ids) per question that leaked, with those chunk ids in the order they were sent, in the order of questions; a question that leaked nothing is left out.
With naive_ask, which trusts the system message to keep runbooks away, a merchant's 13 attack questions in the check (the 8 internal questions of the labelled set and 5 crafted ones) leak at least 10 times. Staff leak nothing. Arjun will ask why the prompt is not security.
What you will learn
- The prompt is not a permission system: why a rule in the system message cannot stop a leak.
- Filters in the search: an audience and a merchant id in the WHERE, before the ranking, and the NULL trap.
- Defence in depth: checking every chunk again before it goes into a prompt.
- Leak tests: attack questions that try to get internal text out.
More in Production
Build "Never leak an internal runbook" yourself
Paisa Setu, a payments company, needs its help centre to answer merchants correctly, with sources. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of RAG in Python are free. This one comes with the all labs plan.