AI invoice reader project (TypeScript) · Chapter 6 of 12
Retries with feedback
By the end of this chapter the reader asks the model again when the rules find problems: it sends the whole conversation back with the model's own answer and a message that says what is wrong, keeps the corrected answer, stops as soon as an answer comes back with the same problems, and adds up what every request cost in tokens. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Checking the model
- 3 tasks
- TypeScript
- In the all labs plan
The first 2 chapters of AI invoice reader project (TypeScript) are free. This one comes with the all labs plan.
Why you need this
Chapter 5's rules, run on the 39 invoices of the inbox that have text, find problems in 18. Twelve of them are the model's own misreadings: dates read month first and plain Totals taken for the total, on Swarnim Sound, Swachh Facility and QuickServe invoices. The paper is right every time; the model just read it the quick way. Five more are on the paper itself, like Joshi Farsan's 6,500 for 50 x 120 and Kavach's GSTIN, and one is an invoice chapter 12 comes back to.
Vaishali could fix the twelve by hand. But the rules already know exactly what is wrong with each: "The invoice date 2026-01-10 is more than 120 days before today", "₹31,100.00 plus ₹5,598.00 of tax is ₹36,698.00, but the total is ₹31,100.00". Tell the model that, and it can read the invoice again with those fields in mind. Vaishali should only see what is still wrong after that.
What you build: 3 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
retry.ts: the feedback, and comparing problems
In src/extract/retry.ts write feedback(problems): the lines "Your JSON does not match the invoice. These checks failed:", then
- ${p.field}: ${p.message}for each problem, then "Read the invoice text again and answer with the corrected JSON for the whole invoice.", joined with "\n". Then same(a, b): true when both lists hold the same set of field and code pairs, whatever their order and their messages.The Check compares your feedback for two problems with its own, then tries same on five pairs of lists.
- Task 2
retry.ts: ask again with the problems
Write extractChecked(ai, text, { instructions, today } = {}): messages = buildMessages(text, instructions); reply = await askModel(ai, messages); check it with checkInvoice(reply.invoice, today). While there are problems and fewer than config.maxAttempts requests were made, make messages a NEW array: the messages so far, then { role: 'assistant', content: reply.content }, then { role: 'user', content: feedback(problems) }; ask again and check again. Add up every reply's usage (addUsage, given); return { invoice, problems, attempts, usage, messages } with the last checked answer.
The Check reads the lab model's request log for swarnim-ssl-2207.pdf (the first answer reads the date month first and the plain Total as the total; one retry fixes both), deccan-dpb-1107.pdf (right the first time) and joshi-jfb-4459.pdf (a line printed wrong: the problem stays).
- Task 3 · interview question
Interview: retry.ts, stop when nothing changes
In extractChecked's loop, stop as soon as an answer's problems are the same (same()) as the problems of the answer before it.
The Check runs joshi-jfb-4459.pdf, whose line 3 is printed 50 x 120 = 6,500: the retry gives the same problem, so it wants exactly 2 requests. Then POST /invoices/read for swachh-sfs-3318.pdf (one retry fixes it).
Nikhil will ask: why does the same problem coming back mean the invoice is wrong, not the model? And what does each retry cost?
What you will learn
- The model remembers nothing: a retry sends the whole conversation again.
- The assistant role: putting the model's own answer back into the conversation.
- Feedback written for the model: the rules' own words, one line per problem.
- Comparing problems as sets of field and code keys.
- When to stop: no problems, config.maxAttempts, or the same problems twice.
- What a retry costs, in tokens.
More in Checking the model
Build "Retries with feedback" yourself
Parda's venues get invoices from sound, printing and canteen vendors, typed and messy. You build the pipeline that reads them, checks every rupee, and sends the doubtful ones to a person. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of AI invoice reader project (TypeScript) are free. This one comes with the all labs plan.