AI booking agent project (TypeScript) · Chapter 12 of 12

Budgets, timeouts and streaming

By the end of this chapter you can limit what one conversation may cost in tokens, pay from the wallet and take in time; stop a model that does not answer; and send the member each step as it happens, over server-sent events. Then you take the whole agent home. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Ready for members
  • 3 tasks
  • TypeScript
  • In the all labs plan

The first 2 chapters of AI booking agent project (TypeScript) are free. This one comes with the all labs plan.

Why you need this

Two people, three numbers, the week before Diwali week opens.

Datta, the token bill. Every step resends the whole conversation (chapter 3), so each step costs more than the one before. One booking, from its trace (kind, name, prompt tokens, completion tokens, ms):

model  lab-chat         770 70 1083
tool   find_showtimes   null null 28
model  lab-chat         898 63 955
tool   book_seats       null null 32
model  lab-chat         1018 46 702
{"modelCalls":3,"toolCalls":2,"toolErrors":0,"promptTokens":2686,"completionTokens":179,"ms":2800}

A conversation where Neha booked and paid twice reached 15,299 prompt tokens in 12 model calls. A loop that goes wrong, as lab-chat-loopy did in chapter 3, keeps paying until the step limit, and nothing adds up the steps of a conversation.

Datta again, the wallet. The agent pays whatever the member says yes to. A cap per conversation keeps one bad afternoon small.

Neha, the wait. A booking takes about 3 seconds, and she sees a spinner until the very end. With lab-chat-slow it is 4 seconds a step, and a model that never answers would hold her request until her phone gives up.

What you build: 3 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    budget.ts: tokens and time

    In src/agent/budget.ts, write Budget.charge(usage): add usage's prompt_tokens and completion_tokens to this.usedTokens; over this.maxTokens, throw new BudgetExceeded('tokens', This conversation has used its ${this.maxTokens.toLocaleString('en-IN')} tokens. Please start a new conversation.). Then Budget.checkTime(): when this.clock() - this.started is more than this.maxSeconds * 1000, throw new BudgetExceeded('time', That took longer than ${this.maxSeconds} seconds, so I stopped. Please try again.).

    The Check charges a budget of 2,000 tokens that has used 300 (1,000 more is 1,300; 700 more is exactly the limit; 1 more throws), then moves a clock of its own to 30 s (fine) and 30.5 s (throws). Then Neha books through the app with agentConfig.maxTokens at 1,200: the second step goes past it, and the answer must be the status over_budget with your sentence.

  2. Task 2

    budget.ts: money

    In src/agent/budget.ts, write Budget.allowPayment(amount): null counts as 0; when this.spent + amount is more than this.maxSpend, throw new BudgetExceeded('money', Paying ₹${amount} would take this conversation past its ₹${this.maxSpend} limit (₹${this.spent} paid so far). Pay on Parda's site instead.); otherwise add amount to this.spent. Then budgetFor(conversationId): a new Budget with agentConfig's maxTokens, maxSpend and runTimeout (as maxSeconds), whose usedTokens is the sum of promptTokens + completionTokens of the conversation's model steps in agentSteps and whose spent is the sum of output ->> 'paid' of its pay_booking tool steps with no error (0 when there are none).

    The Check allows 550 on top of 300 (850), allows null, refuses 200 (and does not count it), then writes a trace of its own (two model steps of 850 and 920 tokens, a pay_booking that paid 550, a failed one and another tool) and wants budgetFor to read 1,770 tokens and ₹550, with the limits from the settings, and 0 and 0 for a new conversation. Then, with maxSpend at 300, Neha books two seats (₹400) and approves paying: the result must be your refusal and the booking still pending.

  3. Task 3 · interview question

    Interview: stream.ts, stream the agent's steps

    In src/agent/stream.ts, write sse(event, data): event: ${event}\ndata: ${JSON.stringify(data)}\n\n. Then StreamHooks, which puts each step on its channel as an [event, data] item: modelStep gives ['model', { tool_calls: [{ name, arguments: argumentsOf(call) }], ms }] for calls or ['model', { text, ms }] for an answer; toolStep gives ['tool', { name, arguments, result, ms }]; waiting gives ['approval', { question }]. Then streamEvents(start), an async generator: start(new StreamHooks(channel)) without awaiting it, and when it settles put ['answer', what it returned] (or ['error', { message: 'The agent stopped: <the error's name>' }] when it threw), then null; meanwhile yield sse(...item) for each item until the null, then sse('done', {}).

    The Check compares your frame, the four kinds of steps on the channel, then streams a start that waits until the first frame has been read (it must arrive within 2 s), then the answer and done, and a start that throws (error, then done). Then POST /agent/chat/stream as Neha must send model, tool, model, tool, model, answer, done.

    Nikhil will ask: why run the turn on its own and pass the steps through a channel, instead of making runAgent itself yield each step?

What you will learn

  • Tokens: what a model step costs, and a budget per conversation checked after every step.
  • Time: a model timeout and a run timeout, and a clock you can pass in.
  • Money: refuse before it moves, and count only what was allowed.
  • A budget that survives requests: read from the trace with sum, coalesce and a JSONB key.
  • Server-sent events: text/event-stream, frames of event: and data: lines.
  • An async generator and a channel: steps sent the moment they happen.

More in Ready for members

Build "Budgets, timeouts and streaming" yourself

Parda members want to book by chat. You build the agent that finds the show, holds the seats through Parda's own API, and asks before it pays. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of AI booking agent project (TypeScript) are free. This one comes with the all labs plan.