LangChain and LangGraph · Chapter 11 of 25

Better retrieval

By the end of this chapter you can tell why a RAG answer went wrong when the search did, get more varied results with MMR, let the model turn a tenant's question into a better search (a metadata filter), split a message with several questions into several searches, and notice when the search found nothing worth sending to the model. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: RAG with LangChain
  • 4 tasks
  • Python
  • In the all labs plan

The first 2 chapters of LangChain and LangGraph are free. This one comes with the all labs plan.

Why you need this

Chapter 9's chain answered most questions right, and its sources showed when it did not: for "What is the late fee for rent?" the retriever brought rules 1.1, 1.3 and 3.3, and the model correctly said it did not know. A RAG chain is only as good as what it retrieves. The model cannot use a rule it was never given, and it will happily use a wrong one.

Mrunal now brings the messages that really arrive. They are not tidy one-line questions:

  • Hinglish: "deposit kab tak wapas milega?". lab-embed was trained mostly on English; Hindi words written in English letters land in odd places.
  • Several questions in one message: rent, guests and food in three lines.
  • Repeats: the FAQ and the policy say the same thing twice, so the top three results can be the same fact three times.
  • Questions the policy does not answer, which still have a "closest" rule.

This chapter adds four small techniques around the retriever, each with pieces you already know.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    mmr.py: MMR versus similarity

    Make one InMemoryVectorStore (lab-embed, as in chapter 8) holding the policy's rules and the FAQ's answers. A rule is a line of /workspace/data/policy.md that starts with a digit: the whole line as page_content, and metadata ref, the rule's number (like 3.1). An answer is a paragraph of /workspace/data/faq.md (the file split at blank lines) that starts with **: the paragraph without spaces or newlines around it as page_content, and metadata ref, the question between the first two pairs of asterisks (like How do I give notice?). Add the rules first, then the answers.

    For the question Moving out: notice and inspection, search with k=3 twice: by similarity, then by MMR with fetch_k=8 and lambda_mult=0.5. Print two lines: Similarity: and the three results' refs joined with a space, a bar and a space ( | ), then MMR: and the MMR results' refs the same way.

  2. Task 2

    pick_section.py: the model writes the search

    Build chapter 8's store of the policy's rules (rule and section in the metadata). Make a Pydantic model with one field, section, whose type is a Literal of the eight sections: Rent, Security deposit, Notice period and lock-in, Guests, Food, Room changes, Maintenance, House rules. Use lab-chat's with_structured_output with it to pick the section of the tenant's question, exactly: deposit kab tak wapas milega?

    Then search the store for the question with k=1 twice: without a filter, and only in the section the model picked. Print three lines: Section: and the section, Without: and the first search's rule number, With: and the second search's rule number.

  3. Task 3

    multi_query.py: one search per question

    Build a store of the policy's rules (each whole rule line, with its number in the metadata as rule). The tenant's message, exactly: Can I change my room for free? When are quiet hours? How long does a Wi-Fi repair take?

    Ask lab-chat to split it: the system message, exactly, Summarize the tenant's message as bullets: the questions they ask. and the message as the human message. Each non-empty line of the reply, without its leading dash and space, is one question.

    Print two lines: One search: and the rule numbers of a k=3 search for the whole message, joined with a comma and a space; then Per question: and, for each question in order, the rule number of a k=1 search, joined the same way, each rule only once.

  4. Task 4 · interview question

    Interview: nothing_found.py, a Hinglish question that finds nothing

    Build a store of the policy's rules (each whole rule line, with its number in the metadata as rule). A tenant writes, exactly: geyser band hai, garam paani nahi aa raha

    Search with scores, k=3, and print one line per result: the rule number, a space, the score with two decimals. Then, if the best score is below 0.3, print Nothing found: passing this to Mrunal; otherwise print Found: and the best rule's page_content.

    The interviewer will ask why this message finds nothing in RoomRaja's policy, and what the assistant should do with such messages.

What you will learn

  • MMR: results that are relevant and different from each other.
  • The model writes the search: structured output picks a section to filter by.
  • Several queries for a message with several questions.
  • When retrieval finds nothing: a score threshold and a handover.

More in RAG with LangChain

Build "Better retrieval" yourself

RoomRaja, a Pune co-living company, answers tenants on WhatsApp by hand. You build its assistant. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of LangChain and LangGraph are free. This one comes with the all labs plan.