Build your own coding agent · Chapter 15 of 20
Budgets and loops
By the end of this chapter you can say what a run costs in tokens and rupees, stop a run at a limit you choose (steps, tokens, rupees, seconds), catch an agent that asks for the same thing again and again, and end a run that cannot succeed with a note a person can pick up. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Scale and trust
- 3 tasks
- Python
- In the all labs plan
The first 2 chapters of Build your own coding agent are free. This one comes with the all labs plan.
Why you need this
MD-111 is the odd one in the tracker. A rider says trip T-4471 cost ₹1 more than the tariff card. Nobody could reproduce it, and T-4471 is not in the October log. There may be nothing wrong in the code at all: a meter misread, a typo on a receipt, a rider who misremembered.
Kaarigar does not know that. Given MD-111, it searches for T-4471, reads meterdown/fare.py, runs the tests (all 44 pass), searches for round_fare, reads meterdown/money.py, lists the files, and then starts again from the top. It never decides to stop: in this lab the model stops by itself only after 60 calls, and real models are no better at admitting "I cannot find it". Every turn round the circle sends the whole conversation again, so each request is bigger than the last, until request 30 is refused:
Error code: 400 - This model's maximum context length is 8192 tokens. However, your messages resulted in 8409 tokens. Please reduce the length of the messages.
Twenty-nine requests were paid for, and the run ends in an error that tells Sneha nothing: not what was tried, not what was ruled out, not what to do next. Kaarigar so far has one limit, max_steps, and it is a crude one: it says nothing about money, it does not notice the agent going round in circles, and when it fires, the run just ends.
Real coding agents have the same problem and the same answers: a limit on turns, a limit on spend, a check for repeated actions, and, when the agent stops, a summary for the person. This chapter builds them.
What you build: 3 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
prices.py: what a run costs
Write prices(): read $OPENAI_BASE_URL/models and return {model id: Price(input, output)} for every model with a price_inr_per_million. Write rupees(usage, price): (usage.prompt × price.input + usage.completion × price.output) / 1,000,000. Under if __name__ == "__main__":, run Kaarigar on a checkout of MD-108 (run_agent with issue_text("MD-108") and Agent(SYSTEM, tools())) and print:
MD-108: N steps, P prompt + C completion tokens
lab-chat: ₹X (rupees at lab-chat's price, 4 decimals)
lab-chat-mini: ₹X (the same usage at lab-chat-mini's price)
- Task 2
budget.py: limits, and a loop detector
Write budget(limits, price=LAB_CHAT), returning (hook, why). hook(run, reply) is an after_step hook: for a reply with no tool calls return None; count each tool call by its name and arguments text, and return "loop" when one is asked for the limits.repeats-th time; return "budget" when run.steps reaches limits.steps, prompt + completion tokens reach limits.tokens, rupees(run.usage, price) reaches limits.rupees, or the seconds since budget() was called reach limits.seconds (a limit that is None is no limit); else None. why() says in a few words what stopped the run ("" until then).
- Task 3 · interview question
Interview: handoff.py, an issue with no fix
Write handoff(run, client=None): one more request to "lab-chat" with run.messages plus a user message HANDOFF and no tools (client defaults to default_client()); add its usage to run.usage and return its text. Under if __name__ == "__main__":, run MD-111 with no budget (max_steps=60) and print no budget: STOP at step N, ₹X. Then run it in a fresh checkout with after_step: the hook of budget(Limits(steps=20, repeats=3)), get the note with handoff(run), and print with a budget: STOP at step N (WHY), ₹X, then the note. (₹X: lab-chat's price, 2 decimals, of run.usage; for the second run after the note.)
What you will learn
- What a run costs: usage per request, prices per million tokens, rupees.
- Budgets: steps, tokens, rupees and seconds, checked after every reply.
- Loop detection: the same call with the same arguments, again.
- The window and the bill are different limits, and both matter.
- The hand-off: one last request, with no tools, for a note a person can use.
More in Scale and trust
Build "Budgets and loops" yourself
MeterDown, a Pune app for auto-rickshaw fares, has eleven open issues. You build Kaarigar, an agent that fixes them without breaking tests, running risky commands or touching files outside the repository. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of Build your own coding agent are free. This one comes with the all labs plan.