RAG: search that answers · Chapter 23 of 37

Chunks that make sense alone

By the end of this chapter you can give every chunk a line of context before it is embedded, measure what that gains, and work out what a model-written context sentence would cost for the whole index. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Better retrieval
  • 4 tasks
  • In the all labs plan

The first 2 chapters of RAG: search that answers are free. This one comes with the all labs plan.

Why you need this

Kabir found the chunk that should answer "what does a refund.failed webhook look like?". It ranks badly, and reading it shows why: it is a JSON payload and two sentences. Nothing in it says "webhook". The words that would, the page title "Webhook events and payload examples" and the heading "refund.failed", were left behind when the page was cut into chunks.

Chapter 20 found the same thing from the other side: the reranker gained nothing until it was given the title. Search has the same problem, earlier, at the moment each chunk is embedded.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    withContext: the context line

    Return the chunk's text with a line of context in front: the title, then ' > ' and the heading when the chunk has one, then a blank line, then the text. A chunk with no heading (heading is '') gets the title alone.

    For help-refund-timelines#2 it is 'How long does a refund take to reach the customer? > When does the clock start?', a blank line, then the chunk's text. For help-refund-timelines#0 it is the title, a blank line and the text. A heading path like 'A > B' stays as it is.

  2. Task 2

    buildContextTable: a table of context embeddings

    Embed withContext(c) for every chunk given, with embedAll, then drop and create TABLE (chunks_ctx) with the columns id text PRIMARY KEY, doc_id text NOT NULL and embedding vector(384) NOT NULL, and insert one row per chunk with its id, doc_id and context embedding. Return the number of rows.

    await buildContextTable(db, chunks) is 866 for the 866 reference chunks, and building it again replaces the table. The embedding stored for help-refund-timelines#2 is the embedding of its context text, not of its text alone. Insert many rows per query (for example 200), not one query per chunk.

  3. Task 3

    contextSearch: search the context embeddings

    Return the ids of the k chunks whose context embedding in TABLE is closest to the query, best first: the same search as vectorSearch, in TABLE.

    contextSearch(db, 'how long is a payment link valid?', 5) is vectorSearch(db, that question, 5, 'chunks_ctx'). On the answerable labelled questions, it puts a right document in the top 5 more often than the plain embeddings.

  4. Task 4 · interview question

    Interview: situate, a sentence of context from the model

    Ask MODEL to situate the chunk: SITUATE as the system message, and one user message with the whole document and the chunk in tags: '<document>', a new line, documentText(c.doc_id), a new line, '</document>', a new line, '<chunk>', a new line, c.text, a new line, '</chunk>'. Return { sentence: the reply's text, trimmed, tokensIn: usage.prompt_tokens }.

    For help-refund-timelines#2 the sentence names its section, 'When does the clock start?'.

    Arjun will ask: one model call per chunk, with the whole document each time. What does that cost for 866 chunks, and is it worth it here?

What you will learn

  • The context line: the title and heading path in front of the chunk's text, for embedding.
  • A second index of context embeddings, and how much it helps, by kind.
  • Situating with a model: a sentence of context written per chunk, as in Anthropic's "contextual retrieval".
  • The cost of doing that for 866 chunks.

More in Better retrieval

Build "Chunks that make sense alone" yourself

Paisa Setu, a payments company, needs its help centre to answer merchants correctly. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of RAG: search that answers are free. This one comes with the all labs plan.