AI invoice reader project · Chapter 4 of 12

Structured outputs

By the end of this chapter you can send an invoice's text to a chat model in one request, ask for an answer in exactly the shape of your Invoice (a structured output), read the reply into an Invoice with its token count, and refuse a reply that is cut off, refused or not an invoice, with one clear sentence. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: From PDF to data
  • 3 tasks
  • Python
  • In the all labs plan

The first 2 chapters of AI invoice reader project are free. This one comes with the all labs plan.

Why you need this

Rafiq typed 186 supplier invoices into the accounts sheet in September, and 9 went wrong. Chapter 1's regex reads the GSTIN and the total, but it read 0 line items: every supplier lays out its table its own way. Chapter 2 gave you each invoice's text and chapter 3 the shape the answer must have. Nothing yet turns the text into that shape.

That is the model's job. But a model's answer is only useful if your code can trust it. Picture the 24-line PackRight invoice coming back cut off in its second line: no subtotal, no taxes, no total. If the app fills in what it can and shows it, Rafiq sees one line of ₹9,675 on an invoice for ₹96,252.60. So this chapter does two things: one request per invoice, and a hard "no" whenever the reply is not a whole invoice.

What you build: 3 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    prompt.py: the two messages

    In app/extract/prompt.py write build_messages(text, instructions=None): a list of two dicts with "role" and "content". The system message holds instructions, or INSTRUCTIONS when none are given; the user message holds "Invoice text:", a blank line, then the text.

    The Check builds the messages for a short text with and without instructions of its own and compares them with its own.

  2. Task 2

    model.py: one request, one Invoice

    In app/extract/model.py write ask_model(ai, messages): one ai.chat.completions.create with model=settings.chat_model, messages, temperature=0, response_format=response_format() and max_completion_tokens=settings.max_output_tokens. Read the first choice's content ("" when None) with read_invoice_json and return Reply(content, invoice, Usage(prompt_tokens, completion_tokens)) from the response's usage. Then extract(ai, text): ask_model with build_messages(text).

    The Check reads the lab model's request log for one of Patel Rice's invoices: one request, the right model, temperature, token limit, response format and messages, and your Reply against the model's reply and usage. Then extract.

  3. Task 3 · interview question

    Interview: model.py, when there is no usable invoice

    In ask_model, before reading the JSON, raise ExtractionFailed with a sentence when the first choice's finish_reason is "length" (cut off) or its message has a refusal. When read_invoice_json raises InvoiceJsonError, raise ExtractionFailed with its first problem in the sentence, from None.

    The Check asks for a 24-line invoice with max_output_tokens 100, uses lab-chat-sloppy (JSON in a code fence), and fakes a refusal and a cut-off reply whose text happens to parse; each must raise ExtractionFailed. Then POST /invoices/extract for an invoice (200) and for the scanned bill (422).

    Kunal will ask: the reply was cut off, but your code could still pull a few fields out of it. Why throw the whole reply away?

What you will learn

  • Messages: what a chat model receives (a list of messages) and what it sends back (one reply). It remembers nothing between requests.
  • The request: model, messages, temperature=0, response_format with a strict json_schema, max_completion_tokens.
  • The response: choices[0].message.content (text, JSON text), finish_reason, refusal and usage (tokens).
  • Tokens: what they are and why you keep the count.
  • When there is no invoice: a reply cut off at the limit, a refusal, and JSON wrapped in a code fence (lab-chat-sloppy).

More in From PDF to data

Build "Structured outputs" yourself

DabbaDrop's kitchens get piles of supplier invoices, typed and messy. You build the pipeline that reads them, checks every number, and sends the doubtful ones to a person. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of AI invoice reader project are free. This one comes with the all labs plan.