RAG in Python · Chapter 11 of 37
Chunks that respect structure
By the end of this chapter you can chunk a Markdown document by its headings, keep every table and code block whole, give each chunk its heading path, and rebuild Paisa Setu's reference index exactly. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Ingestion
- 5 tasks
- Python
- In the all labs plan
The first 2 chapters of RAG in Python are free. This one comes with the all labs plan.
Why you need this
Kabir asked what international cards cost on Growth. The 200-token chunk that held the answer began with " cards | 3.5% | 3.2% | custom |": half a row, and no header to say which column is Growth. Neha has a second complaint: when the assistant cites a chunk, she wants to see where in the document it came from, "Fee table > Worked examples > UPI", not "chunk 7".
Cutting by tokens or by sentences treats a document as a stream of words. Paisa Setu's documents are not: they have headings that say what each part is about, tables whose rows mean nothing without the header, and code that stops working when you cut it. The writer already split each document into meaningful parts. Use their split.
What you build: 5 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
structure.py: blocks, headings, paragraphs, tables and code
Write blocks(body) in structure.py: split body into lines and walk them, collecting lines in a buffer. To flush the buffer as a kind: join its lines with "\n", strip, and add Block(kind=kind, text=that) when it is not empty; then empty the buffer.
A line starting with ``
flushes, then it and every line up to and including the next line starting with`` (or up to the end, when the block is never closed) are one "code" block. A line of 1 to 6 # and a space (the regular expression #{1,6}\s at the start) flushes and adds Block(kind="heading", level=the number of #, text=the line without its #, stripped). A table line starts with | (spaces before it allowed): if the buffer's last line is not a table line, flush first; add the line; if the next line is not a table line, flush as "table". A line with only spaces flushes. Any other line goes into the buffer. Flush once more at the end; everything else is "text".blocks("## Fees\n| A | B |\n| UPI | 0% |\nGST.") is a heading (level 2, "Fees"), a table "| A | B |\n| UPI | 0% |" and the text "GST.". A blank line or a ## line inside a code block stays inside it, and #payouts (no space) is not a heading. For every document, your blocks must be the reference index's.
- Task 2
structure.py: sections, blocks under their heading path
Write sections(items): keep a path of headings and a current section, and walk the blocks. A level 1 heading is the title: skip it. A heading of level L belongs at place L - 2 of the path (counting from 0): cut the path to L - 2 places (a place no heading filled stays empty), put the heading's text at place L - 2, and start a new Section(heading=the path's places that are not empty, in order, parts=[]). Any other block adds its text to the current section's parts; when there is no section yet, start one with heading [].
Return every section in order, even one with no parts: "## Worked examples" then "### UPI" gives Section(heading=["Worked examples"], parts=[]) then Section(heading=["Worked examples", "UPI"], ...). A "### Accepted answer" right after the title, with no ## above it, gives the heading ["Accepted answer"]: a missing level is left out. For every document, your sections must be the reference index's.
- Task 3
structure.py: split_long, cut long sections between blocks
Write split_long(items, max=MAX): for each section, go through its parts collecting them. When something is collected and the collected parts plus this part, joined by "\n\n", would be over max tokens (count_tokens), close a Piece(heading=the section's heading, text=the collected parts joined by "\n\n") and start again with just this part. At the end of the section, close what is left. A part over max on its own stays whole: a table or a code block is never cut. A section with no parts gives no piece. max is MAX (400) when it is not given. Do not change the sections you were given.
With parts of 13, 18, 8, 50 and 5 tokens and max 32, the pieces are 13 and 18 together (31 tokens), then 8, then the 50-token table on its own, then 5.
- Task 4
structure.py: merge_tiny, tiny pieces join the next
Write merge_tiny(pieces, min=MIN, max=MAX): build a new list. For each piece: when the last piece of the new list is under min tokens and its text joined with this piece's text by "\n\n" is at most max tokens, add this text to it that way (it keeps its own heading). Otherwise add a copy of this piece. A merged piece that is still under min keeps taking the next one. At the end, when there are two or more pieces and the last one is under min, remove it and add its text to the one before, joined by "\n\n", even if that goes over max. min is MIN (40) and max is MAX (400) when not given. Do not change the pieces you were given: copy a piece before you add text to it.
With min 10 and max 30, a 2-token "Hi." under Intro takes the next piece ("Hi.\n\nCards cost 1.9% on Starter.", heading ["Intro"]), and a 3-token FAQ at the end joins the piece before it. With tasks 1 to 4 done, chunk_document (given) makes exactly the chunks of data/index/chunks.jsonl for all 145 documents.
- Task 5 · interview question
Interview: structure.py, report over MAX and under MIN
Write report(chunks): a Report with count (how many chunks), median (the size at index len // 2 of the sizes sorted from small to large), max (the largest size), over_max (the ids of the chunks over MAX) and under_min (the ids of those under MIN), both in the order given. Use each chunk's tokens field.
report(read_chunks()) has count 866, median 90 and max 612, four ids over MAX and three under MIN. Arjun will ask: your limit is 400, but your largest chunk is 612 and three are under 40. Bugs?
What you will learn
- Blocks: reading Markdown as headings, paragraphs, tables and code blocks that are never split.
- Heading paths: every chunk knows the headings above it.
- Long and tiny sections: cutting between blocks at 400 tokens, and merging pieces under 40.
- Exceptions: why a few chunks break the limits on purpose.
More in Ingestion
Build "Chunks that respect structure" yourself
Paisa Setu, a payments company, needs its help centre to answer merchants correctly, with sources. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of RAG in Python are free. This one comes with the all labs plan.