RAG: search that answers · Chapter 35 of 37

Old answers to new rules

By the end of this chapter you can make retrieval respect versions and effective dates, answer from the rules that were in force on a given day, show "as of" in every answer, and prove with the labelled questions that the old answers are gone. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Production
  • 5 tasks
  • In the all labs plan

The first 2 chapters of RAG: search that answers are free. This one comes with the all labs plan.

Why you need this

Kabir has a merchant on the phone who was told "money by 6 pm on Wednesday" for a Monday payment. That was the rule until 28 February 2026. Since 1 March settlement is T+1 by 11 am, and the merchant planned their week around the wrong day. The answer was faithful (the last chapter's judge would pass it) and well cited. It was simply quoting a document that is no longer in force.

You cannot just delete the 2024 articles. Neha needs them: a merchant disputing a February settlement must be answered by the February rules. And nothing you have built so far looks at dates. Similarity search finds the text that sounds most like the question, and old text sounds exactly like new text.

What you build: 5 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    isCurrent: in force on a date

    isCurrent(doc, on) is true when the document is in force on the date on: it has started (effective_from is null, or on or before on) and it has not ended (effective_to is null, or on or after on). Both the first and the last day count. Dates are 'YYYY-MM-DD' text, which compares correctly as strings.

    With v2 { effective_from: '2024-07-01', effective_to: '2026-02-28' } and v3 { effective_from: '2026-03-01', effective_to: null }: isCurrent(v2, '2026-10-09') is false, isCurrent(v2, '2026-02-28') is true, isCurrent(v3, '2026-03-01') is true, isCurrent(v3, '2026-02-15') is false, and a document with both dates null is always in force.

  2. Task 2

    searchOn: search only what was in force

    Like search, but return only chunks in force on the date on, with the same rule as isCurrent written in the SQL WHERE, next to the audience condition. Select COLUMNS, so the dates are text, and return the k nearest, nearest first.

    For 'what time of day does the settlement get credited?' on '2026-10-09' it returns 5 chunks, none of them outdated. Filtering after the LIMIT leaves 2, because three of the 5 nearest chunks are from 2024. On '2026-02-15', 'payment received on monday, when will it come to my bank?' returns 5 chunks that include the 2024 articles and none of the v3 ones.

  3. Task 3

    preferLatest: the newest version wins

    When hits from two versions of one document (the same familyOf their doc_id) come back, keep only the chunks of the newest version: the higher version number, or, when both have no version, the later updated date. Keep the hits in their order, and keep every family that has only one version. preferLatest([]) is [].

    The version wins over updated: a 2024 article (version 2) with no end date and updated '2026-05-12' for a typo fix still loses to version 3, updated '2026-03-04'. Of [timeline-2024#2 (v2), where-is-my-money#0, timeline#1 (v3), timeline-2024#4 (v2), timeline#3 (v3), policy-2024#3 (v2)] keep where-is-my-money#0, timeline#1, timeline#3 and policy-2024#3.

  4. Task 4

    answerAsOf: say as of when

    Answer from preferLatest(searchOn(question, audience, on)) with answerFrom and return { text, asOf, citations }. citations are the ids citedIds finds in the reply that were among the chunks sent. asOf is the latest date of the cited chunks, each chunk's effective_from or, when it has none, its updated date; null when nothing is cited. text is 'As of ' + longDate(asOf) + ': ' + the reply, or the reply alone when asOf is null.

    "is there a 6 pm cut-off for a payment to count in today's settlement?" on '2026-10-09' cites help-settlement-cutoff-times#4: asOf '2026-03-01', text 'As of 1 March 2026: ...'. On '2026-02-15', 'what time of day does the settlement get credited?' gives asOf '2024-07-01'. 'saturday ki sales ka paisa kab aayega?' is a refusal: asOf null and the text unchanged.

  5. Task 5 · interview question

    Interview: trapReport, prove the fix

    For the 6 outdated-trap questions (readQuestions() with kind 'outdated-trap'), measure twice and return { before, after }: before with search, after with preferLatest(searchOn(question, audience, on)). Each is { outdatedChunks, staleAnswers, recall }: outdatedChunks adds up the hits not in force on the date (isCurrent), staleAnswers counts the answers (answerFrom with those hits) that cite such a hit, and recall is the share of each question's relevant doc ids found among its hits' doc_ids, averaged over the 6 and rounded to 3 decimals.

    On '2026-10-09' before is about { outdatedChunks: 8, staleAnswers: 3, recall: 0.556 } and after { outdatedChunks: 0, staleAnswers: 0, recall: 0.611 }. Arjun will ask: you changed retrieval, so how do you show it fixed the old answers without breaking anything?

What you will learn

  • Effective dates: when a document is in force, as a rule you can test.
  • Filtering inside the search: why the date filter belongs in SQL.
  • Versions: the newest version wins, even when its dates are wrong.
  • "As of": answers that say which rules they come from, and a before-and-after measurement of the fix.

More in Production

Build "Old answers to new rules" yourself

Paisa Setu, a payments company, needs its help centre to answer merchants correctly. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of RAG: search that answers are free. This one comes with the all labs plan.