AI invoice reader project (TypeScript) · Chapter 4 of 12

Structured outputs

By the end of this chapter you can send an invoice's text to a chat model in one request, ask for an answer in exactly the shape of your Invoice (a structured output), read the reply into an Invoice with the tokens it used, and refuse a reply that is cut off, refused or not an invoice, with one clear sentence. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: From PDF to data
  • 3 tasks
  • TypeScript
  • In the all labs plan

The first 2 chapters of AI invoice reader project (TypeScript) are free. This one comes with the all labs plan.

Why you need this

Vaishali typed 212 vendor invoices into the accounts sheet in September, and 11 went wrong. Chapter 1's regex reads the GSTIN (39 of 39) and the total (38 of 39), but it read 0 line items: Swarnim Sound's table, Joshi Farsan's receipt and Mauli Bakery's typed lines all put their items in a different place. Chapter 2 gave you each invoice's text, and chapter 3 the shape the answer must have. Nothing yet turns the text into that shape.

That is the model's job. But a model's answer is only useful if your code can trust it. Picture Mudran Press's 24-line invoice, mudran-mp-0614.pdf, coming back cut off in its second line: no subtotal, no taxes, no total. If the app fills in what it can and shows it, Vaishali sees one line of ₹7,400 on an invoice for ₹1,47,653.40. So this chapter does two things: one request per invoice, and a hard "no" whenever the reply is not a whole invoice.

What you build: 3 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    prompt.ts: the two messages

    In src/extract/prompt.ts write buildMessages(text, instructions?): an array of two messages. The first is { role: 'system', content: instructions, or INSTRUCTIONS when none are given }; the second is { role: 'user', content: "Invoice text:", a blank line, then the text }.

    The Check builds the messages for a short text with and without instructions of its own and compares them with its own.

  2. Task 2

    model.ts: one request, one Invoice

    In src/extract/model.ts write askModel(ai, messages): one await ai.chat.completions.create({ model: config.chatModel, messages, temperature: 0, response_format: responseFormat(), max_completion_tokens: config.maxOutputTokens }). Read the first choice's message.content ('' when it is null) with readInvoiceJson and return { content, invoice, usage: { input: res.usage.prompt_tokens, output: res.usage.completion_tokens } }. Then extract(ai, text): askModel with buildMessages(text).

    The Check reads the lab model's request log for one of Deccan Popcorn's invoices: one request, the right model, temperature, token limit, response format and messages, and your Reply against the model's reply and usage. Then extract.

  3. Task 3 · interview question

    Interview: model.ts, when there is no usable invoice

    In askModel, before reading the JSON, throw new ExtractionFailed(a sentence) when the first choice's finish_reason is 'length' (cut off) or its message has a refusal. When readInvoiceJson throws an InvoiceJsonError, throw new ExtractionFailed with its first problem in the sentence (any other error goes on as it is).

    The Check asks for a 24-line invoice with config.maxOutputTokens 100, uses lab-chat-sloppy (JSON in a code fence), and fakes a refusal and a cut-off reply whose text happens to parse; each must throw ExtractionFailed. Then POST /invoices/extract for an invoice (200) and for the scanned bill (422).

    Nikhil will ask: the reply was cut off, but your code could still pull a few fields out of it. Why throw the whole reply away?

What you will learn

  • Messages: what a chat model receives (a list of messages) and what it sends back (one reply). It remembers nothing between requests.
  • The request: model, messages, temperature: 0, response_format with a strict json_schema, max_completion_tokens.
  • The response: choices[0].message.content (JSON text), finish_reason, refusal and usage (tokens).
  • Tokens: what they are and why you keep the count.
  • When there is no invoice: a reply cut off at the limit, a refusal, and JSON wrapped in a code fence (lab-chat-sloppy).

More in From PDF to data

Build "Structured outputs" yourself

Parda's venues get invoices from sound, printing and canteen vendors, typed and messy. You build the pipeline that reads them, checks every rupee, and sends the doubtful ones to a person. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of AI invoice reader project (TypeScript) are free. This one comes with the all labs plan.