RAG in Python · Chapter 35 of 37
Old answers to new rules
By the end of this chapter you can make retrieval respect versions and effective dates, answer from the rules that were in force on a given day, show "as of" in every answer, and prove with the labelled questions that the old answers are gone. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Production
- 5 tasks
- Python
- In the all labs plan
The first 2 chapters of RAG in Python are free. This one comes with the all labs plan.
Why you need this
Kabir has a merchant on the phone who was told "money by 6 pm on Wednesday" for a Monday payment. That was the rule until 28 February 2026. Since 1 March settlement is T+1 by 11 am, and the merchant planned their week around the wrong day. The answer was faithful (the last chapter's judge would pass it) and well cited. It was simply quoting a document that is no longer in force.
You cannot just delete the 2024 articles. Neha needs them: a merchant disputing a February settlement must be answered by the February rules. And nothing you have built so far looks at dates. Similarity search finds the text that sounds most like the question, and old text sounds exactly like new text.
What you build: 5 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
freshness.py: is_current, in force on a date
Kabir's merchant was told the 2024 rule. Write is_current(doc, on) in freshness.py: True when the document is in force on the date on, which means it has started (effective_from is None, or on or before on) and it has not ended (effective_to is None, or on or after on). Both the first and the last day count. The dates are datetime.date objects.
With v2 (effective_from 2024-07-01, effective_to 2026-02-28) and v3 (effective_from 2026-03-01, effective_to None): is_current(v2, date(2026, 10, 9)) is False, is_current(v2, date(2026, 2, 28)) is True, is_current(v3, date(2026, 3, 1)) is True, is_current(v3, date(2026, 2, 15)) is False, and a document with both dates None is always in force.
- Task 2
freshness.py: search_on, search only what was in force
Write search_on(conn, question, on, audience="merchant", k=5, table="chunks"): like search, but return only chunks in force on the date on, with the rule of is_current written in the SQL WHERE, next to the audience condition, and on passed as a parameter. Select COLUMNS and return the k nearest as Hits (as_hits), nearest first.
For "what time of day does the settlement get credited?" on 2026-10-09 it returns 5 chunks, none of them outdated; filtering after the LIMIT leaves 2, because three of the 5 nearest chunks are from 2024. On 2026-02-15, "payment received on monday, when will it come to my bank?" returns 5 chunks that include the 2024 articles and none of version 3.
- Task 3
freshness.py: prefer_latest, the newest version wins
Write prefer_latest(hits): when hits from two versions of one document (the same family_of their doc_id) come back, keep only the chunks of the newest version: the higher version number, or, when the versions are equal (both None, say), the later updated date. Keep the hits in their order, and keep every family that has only one version. prefer_latest([]) is [].
The version wins over updated: a 2024 article (version 2) updated 2026-05-12 for a typo fix still loses to version 3, updated 2026-03-04. Of [timeline-2024#2 (v2), where-is-my-money#0, timeline#1 (v3), timeline-2024#4 (v2), timeline#3 (v3), policy-2024#3 (v2)] keep where-is-my-money#0, timeline#1, timeline#3 and policy-2024#3.
- Task 4
freshness.py: answer_as_of, say as of when
Write answer_as_of(conn, question, on, audience="merchant", table="chunks"): answer with answer_from from prefer_latest(search_on(conn, question, on, audience, 5, table)), and return AsOf(text, as_of, citations). citations are the ids cited_ids finds in the reply that were among the chunks sent. as_of is the latest date of the cited chunks, each chunk's effective_from or, when it has none, its updated date; None when nothing is cited. text is "As of " + long_date(as_of) + ": " + the reply, or the reply alone when as_of is None.
"is there a 6 pm cut-off for a payment to count in today's settlement?" on 2026-10-09 cites help-settlement-cutoff-times#4: as_of date(2026, 3, 1), text "As of 1 March 2026: ...". On 2026-02-15, "what time of day does the settlement get credited?" gives as_of date(2024, 7, 1). "saturday ki sales ka paisa kab aayega?" is a refusal: as_of None and the reply unchanged.
- Task 5 · interview question
Interview: freshness.py, trap_report, prove the fix
Write trap_report(conn, on, table="chunks"): for the 6 outdated-trap questions (read_questions() with kind "outdated-trap"), measure twice and return TrapReport(before, after): before with search, after with prefer_latest(search_on(...)) on the date on, each question with its own audience and k 5. Each is TrapNumbers: outdated_chunks adds up the hits not in force on the date (is_current), stale_answers counts the answers (answer_from with those hits) that cite such a hit, and recall is the share of each question's relevant doc ids found among its hits' doc_ids, averaged over the 6 and rounded to 3 decimals.
On 2026-10-09 before is outdated_chunks 8, stale_answers 3, recall 0.556 and after is 0, 0 and 0.611. Arjun will ask: you changed retrieval, so how do you show it fixed the old answers without breaking anything?
What you will learn
- Effective dates: when a document is in force, as a rule you can test.
- Filtering inside the search: why the date filter belongs in SQL.
- Versions: the newest version wins, even when its dates are wrong.
- "As of": answers that say which rules they come from, and a before-and-after measurement of the fix.
More in Production
Build "Old answers to new rules" yourself
Paisa Setu, a payments company, needs its help centre to answer merchants correctly, with sources. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of RAG in Python are free. This one comes with the all labs plan.