AI ordering agent project · Chapter 7 of 12

Human approval

By the end of this chapter your agent stops before it moves money: it asks the member "Shall I pay ₹550 for order 2 (2 x Jain Thali) from your wallet? Answer yes or no.", keeps the waiting call in Postgres until the answer comes, runs it on yes and tells the model no on no, and pays once even when the member taps Yes twice in the same second. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Remembering
  • 3 tasks
  • Python
  • In the all labs plan

The first 2 chapters of AI ordering agent project are free. This one comes with the all labs plan.

Why you need this

One of the 19 wrong orders in the pilot week was a payment taken twice: a member pressed Pay again because the first tap felt slow, and Rafiq spent an afternoon refunding it. Farah's rule for the agent is short: it never takes money without the member's yes, and one yes pays once.

The agent from chapter 6 cannot keep that rule. Since chapter 3 the loop runs every call the model asks for, and pay_order is one of them. "Pay for it from my wallet" makes the model ask for pay_order, the loop runs it, and the money is gone before anybody was asked. A sentence in the system message ("always ask first") would not change that: the model writes calls, your loop runs them. The stop has to be in your code.

What you build: 3 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    approval.py: which calls wait, and what to ask

    In agent/approval.py, write needs_approval(call): True when call.name is in APPROVAL_TOOLS, else False. Then question(ctx, pending): pending is a list of {"id", "name", "arguments"} whose arguments hold an order_id. Look the member's orders up with ctx.api.get("/orders/me")["orders"]. For each call say "pay ₹<total> for order <id> (<quantity> x <name>, ...) from your wallet", or "pay for order <id>" when the order is not in the list; join them with " and " and return "Shall I <them>? Answer yes or no."

    The Check asks needs_approval about pay_order (True) and place_order, find_dish and list_my_orders (False), asks your question about Tara's pending order of 2 Jain Thali ("Shall I pay ₹550 for order <id> (2 x Jain Thali) from your wallet? Answer yes or no.") and about order 1, Advait's. Then, through the app, Tara books and says "Pay for it from my wallet": the agent must stop with status needs_approval and your question, and nothing may be paid.

  2. Task 2

    approval.py: wait, then answer the calls

    In agent/approval.py, write pause(db, conversation_id, pending, ask): set the conversation's pending column to {"calls": pending, "question": ask} and commit. Then answers_for(ctx, pending, approved, reason=DECLINED): a tool message for each call of pending["calls"], in order, {"role": "tool", "tool_call_id": call["id"], "content": json.dumps(result, ensure_ascii=False)}, where result is run_tool(ctx, call["name"], call["arguments"]) when approved and {"error": reason} when not.

    The Check pauses a conversation (the pending column, committed), answers a pay_order call with no (the DECLINED error, then a reason of its own, and the order still pending) and with yes (the tool's result, and the order confirmed).

  3. Task 3 · interview question

    Interview: approval.py, answered once

    In agent/approval.py, write take_pending(db, conversation_id): lock the conversation's row (select with with_for_update()), take its pending (set the column to None) and commit, and return what was pending, or None when nothing was waiting. On any error roll back and raise it again.

    The Check takes a pending (returned, then gone; a second call gives None). Then a transaction of its own locks the conversation, takes the calls and stays open while your take_pending runs as the second Yes; it commits, and your call must return None. Then, through the app, Tara books, asks to pay and sends two POST /agent/approve at the same moment: one 200, one 409, and exactly one payment.

    Kunal will ask: Tara taps Yes twice, and two requests reach the API in the same second. How do you make sure her wallet pays once?

What you will learn

  • Human approval: which calls wait for a person, decided by the tool's name.
  • The question: what the member is agreeing to, in plain words and rupees.
  • Pausing a run: the waiting calls kept in a JSON column, so the answer can come minutes later in another request.
  • Answering every call: a tool message for each call id, on yes and on no, and the 400 when one is missing.
  • Two Yes taps at once: the race that pays twice, for real, and the row lock that stops it.

More in Remembering

Build "Human approval" yourself

DabbaDrop members want to order by chat. You build the agent that books their tiffins through DabbaDrop's own API, and asks before it pays. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of AI ordering agent project are free. This one comes with the all labs plan.