Build an AI gateway · Chapter 15 of 21
Budgets in rupees
By the end of this chapter you can keep each of Jawab's brands within its daily budget in rupees, explain why "check, then spend" overspends the moment requests arrive together, reserve the worst case and settle the real cost, move a brand to cheaper models as it nears its limit, and reconcile Darwaza's own sums with the providers' bill. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Spend less
- 3 tasks
- Python
- In the all labs plan
The first 2 chapters of Build an AI gateway are free. This one comes with the all labs plan.
Why you need this
Lakshmi has sold each brand a plan: so many rupees of AI a day. Nila Handlooms pays for ₹2.50, Payyoli Snacks for ₹8 (lab teams). Today nothing stops a brand from spending ten times that: a bug in Nila's app that retries every reply in a loop, or a sale bigger than planned, and Jawab pays the difference. She wants three things: no brand ever past its budget, a warning before it gets there, and Darwaza's numbers to match the providers' invoices.
Chapter 14 limited how fast a brand may send. This chapter limits how much it may spend. A slow, steady stream of expensive replies passes every rate limit and can still empty a budget by lunch.
What you build: 3 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
budgets.py: a budget in rupees per team
Write class Budgets: __init__(self, limit_of=a function giving each team's daily_budget_inr from TEAMS, or math.inf), keeping self.limit_of, self.spent and self.reserved (dictionaries of team to rupees); used(team) is spent + reserved; level(team) is "ok" under 80% of the limit, "tight" from 80%, "over" from 100%; reserve(team, inr) holds inr and returns True only if used + inr does not pass the limit (otherwise holds nothing and returns False); settle(team, reserved_inr, actual_inr) takes reserved_inr off reserved and adds actual_inr to spent.
- Task 2
gateway.py: Darwaza within budget
At the top level, budgets = Budgets() and darwaza = create_darwaza(). Write async def handle(req): level "over" returns {"ok": False, "status": 402, "kind": "budget", "error": ...} without calling anything; level "tight" sends a copy of the request with "model" set to the cheapest target of route(req) (by PRICES[t].input + PRICES[t].output); reserve the highest worst_case_inr over route(req) (402 when it does not fit); await darwaza.handle, settle with the answer's cost_inr (0 when it failed) and return the answer.
Under if __name__ == "__main__":, replay the whole day through handle with ledger() before and after, print the report, then per team of TEAMS and a last line:
kaapi: ₹S of ₹B, level
Darwaza's sum: ₹A, the providers' bill: ₹C
- Task 3 · interview question
Interview: race.py, ten requests, one budget
With Nila's first summary (day(features=["summary"], teams=["nila"])[0].request) on "kestrel/kestrel-swift" and a limit of ₹0.04, send ten at once (asyncio.gather) twice. First check then spend (skip if spent >= 0.04, else call and add the cost after), then reserve then settle (Budgets(lambda team: 0.04): reserve worst_case_inr, call, settle). Print:
check then spend: N calls, ₹X of ₹0.04
reserve then settle: N calls, ₹X of ₹0.04
(4 decimals; the second X is budgets.used("nila") at the end.)
What you will learn
- Spend per team: rupees from each answer's usage, added up per brand.
- The race: why "if spent < limit, call" lets ten requests through at once.
- Reserve, then settle: hold the worst case before a call, swap in the real cost after.
- The worst case: what a request could cost, wherever its route ends up.
- Levels: under 80% fine, from 80% cheaper models first, at 100% Darwaza's own 402.
- Reconciling: Darwaza's sums against the providers' ledger, and why they differ a little.
More in Spend less
Build "Budgets in rupees" yourself
Jawab, a Kochi support desk, answers customers for six Kerala brands on three AI providers that fail, slow down and cost different amounts. You build Darwaza, its gateway, and replay a whole day of traffic against an SLO and a budget. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of Build an AI gateway are free. This one comes with the all labs plan.