LangChain and LangGraph · Chapter 7 of 25

Documents and splitters

By the end of this chapter you can say what RAG is and why it starts with cutting documents into pieces, turn RoomRaja's policy and FAQ into LangChain Documents with metadata, split them by their headings with MarkdownHeaderTextSplitter and by size with RecursiveCharacterTextSplitter, choose a chunk size by measuring it, and explain why a rule cut in half makes the model answer wrongly. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: RAG with LangChain
  • 4 tasks
  • Python
  • In the all labs plan

The first 2 chapters of LangChain and LangGraph are free. This one comes with the all labs plan.

Why you need this

In chapter 1 the model told a tenant that all three meals are included and that notice is 15 days on WhatsApp. Both wrong: it had never seen RoomRaja's policy. The fix was to give the model RoomRaja's facts in the request.

The simplest way is to paste the whole policy into the system message. That works today: policy.md is 2,188 characters, and a question with the whole policy pasted in cost 629 input tokens in the lab. But Tanvi's plans do not stop at one policy page. There is the FAQ, the house notes of six properties, the rent rules of every year, the maintenance manual. Pasted into every one of 120 tenants' messages, that is slow and expensive, and soon it does not fit in the model's window at all.

So the assistant should send the model only the part that answers the question. That is RAG, retrieval-augmented generation: first retrieve the few pieces of your documents that fit the question, then let the model generate its answer from them. This class builds it in five chapters. This one is the first step: turning RoomRaja's files into pieces worth finding.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    load_docs.py: RoomRaja's files as Documents

    Sid wants RoomRaja's two documents in LangChain's own shape before anything is searched. Read /workspace/data/policy.md and then /workspace/data/faq.md, and make one Document for each: the file's whole text as its page_content, and metadata with two keys, source (the file's name, like policy.md) and title (the file's first line without the "# " in front).

    Then go through your list of Documents and print one line per Document, from its metadata and its text, written as: policy.md: RoomRaja tenant policy, N characters (N is the length of its page_content).

  2. Task 2

    by_headers.py: one chunk per section

    Split /workspace/data/policy.md by its headings with MarkdownHeaderTextSplitter, splitting on # (stored as title) and ## (stored as section).

    Print the number of chunks as N chunks, then one line per chunk, in order: the chunk's section from its metadata (or (none) when it has no section), a colon and a space, and how many rules the chunk holds, as 3 rules. A rule is a line of the chunk that starts with a digit.

  3. Task 3

    chunk_size.py: choose a chunk size by measuring

    Faisal wants a number, not a guess. The policy's rules are the lines of /workspace/data/policy.md that start with a digit. For each chunk size 60, 100, 140 and 300, in that order, split the policy with RecursiveCharacterTextSplitter (chunk_overlap 0) and count the rules that are whole in some chunk (the rule's line appears in it, unchanged).

    Print one line per size: 60: N chunks, W of R rules whole (R is the number of rules). Then a last line: Smallest size with every rule whole: and that size, worked out from your measurements.

  4. Task 4 · interview question

    Interview: cut_rule.py, half a rule

    Split /workspace/data/policy.md with RecursiveCharacterTextSplitter, chunk_size 60 and chunk_overlap 0, and take the chunk that starts with 4.2 (the first half of rule 4.2). Take the whole rule too: the line of the file that starts with 4.2.

    Ask lab-chat the same question twice, exactly: Do I need approval for an overnight guest? Each time send a system message, Answer only from the policy below., a blank line, a line Policy:, and then the text (first the half, then the whole rule), followed by the question as the human message.

    Print two lines: Half: and the first reply, then Whole: and the second reply.

    The interviewer will ask why the half rule answers wrongly, and what that says about choosing a chunk size.

What you will learn

  • Document: LangChain's piece of text with metadata.
  • Loading files by hand into Documents.
  • MarkdownHeaderTextSplitter: one piece per heading.
  • RecursiveCharacterTextSplitter: pieces of a chosen size, with overlap.
  • Measuring a chunk size instead of guessing it.

More in RAG with LangChain

Build "Documents and splitters" yourself

RoomRaja, a Pune co-living company, answers tenants on WhatsApp by hand. You build its assistant. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of LangChain and LangGraph are free. This one comes with the all labs plan.