AI invoice reader project · Chapter 2 of 12
Reading PDFs
By the end of this chapter you know what a PDF holds inside, you can read every page of an invoice into text that keeps its columns, refuse a file that is too big, too long or not a PDF at all, and tell a PDF with text from a photo of a paper bill. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: From PDF to data
- 3 tasks
- Python
- Free chapter
Free with a Google or GitHub sign-in, like the first 2 chapters of every lab.
Why you need this
From chapter 4 on, the model reads DabbaDrop's invoices. A model reads text, so the text you hand it is all it will ever know about the invoice. If the text runs two columns together, mixes up which number belongs to which line, or is empty, no model can put it right.
Chapter 1's quick_text was good enough for two regexes. Look at what it made of Shree Ganesh's vegetable bill:
GSTIN: 27ABJPG3317K1ZN Phone: +91 98201 44718
BILL OF SUPPLYInvoice No: SG/1041
Date: 02/10/2026
The bill prints "Invoice No: SG/1041" on the left and "BILL OF SUPPLY" on the right of the same line. Quick reading glued them into one word, in the order they were drawn. And for Kisan Fresh Farms' paper bill, which Meera photographed and sent as a PDF, it gave '': nothing at all. The reader must keep each line's columns apart, say which page a line came from (PackRight's 24-line order runs over two pages), refuse a 40 MB forward or a text file renamed .pdf before wasting work on it, and notice the photo, so that Rafiq looks at it instead of a model guessing.
The lesson
Tour of the codebase
New or changed since chapter 1:
app/pdf/reader.py yours: PdfText (given, two properties to write),
clean_page (Task 1), read_pdf (Task 2)
app/modules/reading/router.py given: POST /invoices/text {"file"}
app/modules/files.py given: FileIn, and invoice_text (chapter 4 uses it:
the text, or 422 needs_ocr)
app/main.py changed: includes the reading router
app/inbox.py, app/baseline.py now given: chapter 1's code
POST /invoices/text takes {"file": "ganesh-sg-1041.pdf"}, finds the file with chapter 1's inbox_path (which refuses a name outside the inbox), and answers with the page count, the number of characters, needs_ocr and the text. Before you write anything it answers 500, and the log ends with NotImplementedError: Task 2: read_pdf is not written yet.
What a PDF holds
A PDF is not a picture of a page and not a text file. It is a list of drawing commands. DabbaDrop's invoices are plain ASCII on purpose, so you can look inside one with head:
$ head -c 400 data/inbox/kisan-kf-318.pdf
%PDF-1.4
1 0 obj
<< /Type /Catalog /Pages 2 0 R >>
endobj
2 0 obj
<< /Type /Pages /Kids [7 0 R] /Count 1 >>
endobj
3 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>
endobj
...
Numbered objects: a catalog, a list of pages, the fonts. Further down, the page's content stream draws the invoice:
BT /F2 15 Tf 1 0 0 1 40.0 790.0 Tm (Kisan Fresh Farms) Tj ET
BT /F2 11 Tf 1 0 0 1 40.0 726.0 Tm (Bill No. KF-318) Tj ET
BT /F1 11 Tf 1 0 0 1 300.0 726.0 Tm (Dt. 3.10.26) Tj ET
BT /F1 10 Tf 1 0 0 1 50.0 682.0 Tm (1\) Tamatar - 40 kg x 32 = 1,280) Tj ET
Each line says: use font F2 at size 15, move to x 40, y 790 (from the bottom left), and draw this text. There are no lines, columns or tables in a PDF, only pieces of text placed at points. "Bill No. KF-318" and "Dt. 3.10.26" are two separate pieces at the same height, x 40 and x 300. A reader that wants lines has to rebuild them from the positions.
The photographed bill is different. Its page draws one thing:
q 595 0 0 842 0 0 cm /Im1 Do Q
"Paint image Im1 over the whole page." The image is 200 by 283 grey pixels. No characters anywhere: the words "Tamatar" and "Total" exist only as shapes in those pixels. Reading them needs OCR (optical character recognition), which this project does not do.
Plain mode and layout mode
pypdf reads PDFs in Python. PdfReader(path) opens one, .pages is the list of its pages, and page.extract_text() gives a page's text. That default, plain mode, writes the pieces in the order they were drawn, which is how "BILL OF SUPPLY" and "Invoice No: SG/1041" became one word: the bill drew them one after the other, with nothing between.
extract_text(extraction_mode="layout") uses the positions instead. It puts each piece at its place on a grid of characters, the way the page looks:
Invoice No: SG/1041 BILL OF SUPPLY
Date: 02/10/2026
...
S.No Item Qty Unit Rate Amount
1 Pyaz (Onion) 60 kg 28.00 1,680.00
2 Aloo (Potato) 50 kg 24.00 1,200.00
Every quantity sits under Qty and every amount under Amount. For an invoice that is the whole point: the model can tell which number is the rate and which the amount. The Check compares your pages with layout mode's, and if it finds the glued words it says so:
read_pdf's text runs "BILL OF SUPPLYInvoice No" together: use extract_text(extraction_mode="layout"), which keeps the columns
Tidying a page
Layout mode pays for its grid. It keeps the page's empty space too: on Ganesh's bill there are 54 newlines in a row between the Total and the bank details, where the paper is blank. Other PDFs bring spaces at the ends of lines and non-breaking spaces ("\xa0", a space a program told not to break a line at, which is not " "). The model would read all of it, and a model is paid for by how much it reads. clean_page tidies a page without moving a column.
str.replaceswaps one character for another.str.rstrip()removes spaces at the right end only. The spaces at the start of a line are the columns: they must stay.- A regex count with no upper end finds a run of any length:
{2,}means "two or more".
>>> re.sub(r" {2,}", " ", "Pyaz (Onion) 60 kg")
'Pyaz (Onion) 60 kg'
(That is the kind of tidying that would break a table, so it is not yours to do; the same {n,} idea on newlines is.) Ganesh's page goes from 2,078 characters to 2,026. The trap is the last step: strip() removes the newlines at the ends, and also the spaces of the first line. The Check catches it:
clean_page gave "1 Pyaz 60\n\n 2 Aloo 50" for spaces at the start of lines (the columns); the task wants " 1 Pyaz 60\n\n 2 Aloo 50"
str.strip("\n") removes only the characters you name.
Refusing early
Three files should never reach the model, and each has an error code the API sends back.
- Too big (
413 too_large). A kitchen manager forwards a 40 MB scanned catalogue.path.stat().st_sizeasks the file system for the size without reading a byte, so check it first:settings.max_bytesis 5,000,000. - Not a PDF (
422 not_a_pdf). pypdf reads lazily: a broken file can fail when you open it or when you first ask for its pages, so both go inside onetry. On a text file named.pdf, pypdf logsinvalid pdf header: b'This 'and raisesPdfStreamError: Stream has ended unexpectedly, a kind ofPdfReadError. CatchPdfReadError(andValueError, which some broken files raise) and raise your ownAppErrorwithfrom None. - Too long (
422 too_many_pages).settings.max_pagesis 10: an invoice is a page or two, and a 300-page PDF is something else.
Each AppError carries a sentence a person can read. From the finished chapter, with smaller limits so the real invoices trip them:
422 not_a_pdf 02-reading-pdfs-note.pdf is not a PDF pypdf can read: Stream has ended unexpectedly
422 too_many_pages packright-prc-7765.pdf has 2 pages; the most is 1
413 too_large kisan-kf-318.pdf is 2,153 bytes; the most is 1,000
Pages and scans
PdfText is a frozen dataclass holding one string per page. Its two other values are properties: a method marked @property is read like a field (pdf.text, no brackets) and computed from pages each time.
textputs a marker line before every page,--- page 1 of 2 ---, and a blank line between pages. When PackRight's order runs onto a second page, the model (and Rafiq, later) can see where page 2 begins.enumerate(pages, 1)numbers from 1, the way people count:[(1, 'Andheri'), (2, 'Dadar'), (3, 'Thane')].needs_ocris True when no page has any text. Careful: a page of only spaces is not an empty string, soany(["", " \n "])isTrue. Strip each page first:any(s.strip() for s in ["", " \n "])isFalse.
The finished route, as Meera:
{"file":"kisan-kf-scan-0930.pdf","pages":1,"chars":20,"needs_ocr":true,"text":"--- page 1 of 1 ---\n"}
[200]
{"file":"ganesh-sg-1041.pdf","pages":1,"chars":2046,"needs_ocr":false,"text":"--- page 1 of 1 ---\nShree Ganesh Vegetable Suppliers\n\nShed 14, APMC Market, ..."}
[200]
{"error":{"code":"bad_file","message":"'../app/config.py' is not a PDF in the inbox"}}
[400]
The given invoice_text in app/modules/files.py is what chapter 4 sends the model: it raises 422 needs_ocr for a scan, so a photo never reaches the model at all.
Your turn
- Task 1, clean_page: replace,
rstripeach line, collapse runs of blank lines, strip only newlines at the ends. The trap:strip(). - Task 2, read_pdf: the size first, then pypdf inside a
try, the page count, then each page in layout mode, cleaned. The trap: plain mode. - Task 3, PdfText:
textwith markers,needs_ocr. Kunal will ask: the scanned bill gives you no text at all. Why not send it to the model anyway, and what happens to it instead?
Start the app with python -m app and send "The text of a vegetable bill", "A 2-page invoice" (find the second marker), "The scanned bill" and "A file outside the inbox". Try head -c 400 on a PDF in data/inbox, then press Check.
What you build: 3 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
reader.py: tidy one page's text
In app/pdf/reader.py write clean_page(raw): every non-breaking space (" ") as a plain space, the spaces at the end of each line removed, any run of 3 or more newlines made 2, and the newlines at the very start and end removed. Spaces at the start of a line stay.
The Check cleans five texts: non-breaking spaces, spaces at line ends, runs of blank lines, indented columns, and lines of only spaces.
- Task 2
reader.py: read a PDF, page by page
Write read_pdf(path). A file bigger than settings.max_bytes: raise AppError(413, "too_large", ...). Open it with PdfReader(path) and get its pages; if pypdf raises PdfReadError or ValueError, raise AppError(422, "not_a_pdf", ...). More pages than settings.max_pages: raise AppError(422, "too_many_pages", ...). Return PdfText with each page's page.extract_text(extraction_mode="layout") cleaned with clean_page.
The Check reads a 2-page invoice and compares each page with its own reading, then reads a text file named .pdf, a PDF over a smaller max_bytes and the 2-page PDF with max_pages 1.
- Task 3 · interview question
Interview: reader.py, page markers and scans
Write PdfText's two properties. text: every page after a marker line "--- page 1 of 2 ---" and a newline, the pages joined with a blank line between them. needs_ocr: True when no page has any text that is not spaces, False otherwise.
The Check builds PdfText objects of its own, then calls POST /invoices/text for the scanned bill (kisan-kf-scan-0930.pdf, needs_ocr true) and for ganesh-sg-1041.pdf (its text starts with the marker, then the supplier's name).
Kunal will ask: the scanned bill gives you no text at all. Why not send it to the model anyway, and what happens to it instead?
What you will learn
- What a PDF holds: drawing commands that place characters on a page, and why a scan holds only pixels.
- pypdf: PdfReader, its pages, and extract_text in plain and layout mode.
- Tidying text with string methods and a regex with a count.
- Refusing early: a size limit before reading, and pypdf's errors turned into the API's errors.
- Properties on a dataclass: values computed from its fields.
- Text or no text: how the app knows a PDF needs OCR.
More in From PDF to data
Build "Reading PDFs" yourself
DabbaDrop's kitchens get piles of supplier invoices, typed and messy. You build the pipeline that reads them, checks every number, and sends the doubtful ones to a person. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
Free with a Google or GitHub sign-in, like the first 2 chapters of every lab.