Harness engineering · Chapter 17 of 23

Subagents

By the end of this chapter you can hand a search to a subagent: a second loop with its own small context, tools that only look, and a cheaper model, which returns a short report instead of everything it read. You will measure what it saves the main agent on T10, and say when it is worth it and when a plain search is cheaper. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Scale and measure
  • 4 tasks
  • Python
  • In the all labs plan

The first 2 chapters of Harness engineering are free. This one comes with the all labs plan.

Why you need this

T10 is a rename. Last week gst.rate_for became gst.hsn_rate, and three modules still call the old name, so importing them fails. Fixing it is six one-line edits. Finding the six lines is the hard part.

Kabir gave the main agent the tools that change code: read_file, list_files, edit_file and run_command. Run on T10, it lists the files, reads gst.py (two pages), then dates.py, invoice.py, money.py, quote.py, reminders.py and reports.py, one after another, looking for rate_for. Every file it reads stays in its conversation and is sent again with every later request. Request 15 is 8,158 prompt tokens; request 16 would be 8,249, and the lab model refuses it:

Error code: 400 - This model's maximum context length is 8192 tokens.
However, your messages resulted in 8249 tokens.

The six edits happened to land before it died, so the grade passes this time, but the run never ran the tests and never answered. One more caller in one more file and the job would have been left half done. Most of what filled the window was looking: 192 rows of HSN codes, holiday dates, reminder text, none of it needed for the edits. What the main agent needed from all that reading was six lines.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    subagent.py: a helper with its own context

    Kabir wants the looking done somewhere else, so the main agent's context stays small. In subagent.py, write HELPER_SYSTEM: a system prompt saying it is a helper for Hisaab's coding agent, that it only looks (searches and reads, never changes the code), and that it answers with what it found, one place per line, as path:line: code.

    Then write run_subagent(root, task, max_steps=6). It runs run_agent (harness.py, given) with the task as the user's message and HELPER_SYSTEM as the system prompt, nothing else; only read_file, list_files and search_code from repo_tools(root); on lab-chat-mini, at most max_steps requests. It returns {"report": the helper's text answer, or None when it did not answer, "steps": the requests it sent, "prompt_tokens": their prompt tokens added up}.

    Under if __name__ == "__main__":, run it on a checkout of T10 with the task Find every place in the code that uses rate_for. Report each one as path:line: code. and print the report, then a last line like:

    helper: 3 requests, 900 prompt tokens

    Trap: a helper that changes the checkout, or one offered edit_file or run_command, is not a helper.

  2. Task 2

    delegate_tool.py: the delegate tool

    In delegate_tool.py, write delegate_tool(root, runs=None): it returns a Tool named delegate with one required parameter, task (a string), described as giving a search task to a helper with its own context, which looks through the code and reports back what it found.

    Its function runs run_subagent(root, task) from your subagent.py, appends the helper's dictionary to runs when runs is a list, and returns the report. A report over 2000 characters comes back as its first 2000 characters, then a line [report cut: N more characters]. With no report, it returns the helper stopped after N requests without a report.

    Under if __name__ == "__main__":, on a checkout of T10, run run_agent with T10's issue and read_file, list_files, edit_file, run_command and your delegate tool (no search_code: the looking is the helper's job). Print the answer, then grade the checkout and print T10: passed or T10: failed.

  3. Task 3

    t10_tokens.py: what the helper saves

    Nandini wants the number. In t10_tokens.py, run T10 twice, each in a fresh checkout, with run_agent and T10's issue: reading, with read_file, list_files, edit_file and run_command; then delegating, with those four and your delegate tool. For each run print one line, reading first:

    reading: passed, stop answered, 9 requests, 7000 main prompt tokens, largest 1200, helper 0

    (those numbers only show the format): passed or failed from the grade, the run's stop reason, the main agent's requests, its prompt tokens added up, the largest of them, and the helper's prompt tokens added up (0 when nothing was delegated).

    Trap: result.usage holds only the main agent's requests; the helper's tokens come from the dictionaries delegate_tool appends to runs.

  4. Task 4 · interview question

    Interview: handed_all.py, a helper handed everything

    A teammate changed the delegate tool "so the helper knows the whole story": make_delegate now hands the helper the main agent's conversation so far, with the task added at the end. Since then T10 fails. Run handed_all.py, find what goes wrong, and fix make_delegate so that T10 passes again, with a helper on lab-chat-mini, offered read_file, list_files and search_code only, that starts from its own two messages (HELPER_SYSTEM, then the task the main agent wrote) and returns its report as the result.

    Keep make_delegate(root, conversation), run(root, issue) and HELPER_SYSTEM, and what each returns; run() keeps passing its conversation to make_delegate.

What you will learn

  • What a subagent is: the same loop, a fresh context, different tools, a different model.
  • Why its tools only look, and why it runs on lab-chat-mini.
  • The delegate tool: the main agent writes a task, the helper's report comes back as the result, cut short when it is long.
  • What it saves, measured on T10, and when a search tool is cheaper still.
  • Why a helper starts fresh, and what happens when it is handed the main agent's conversation.

More in Scale and measure

Build "Subagents" yourself

Hisaab, a Lucknow invoicing startup, wants a coding agent to clear its bug list. You build its harness and score it on twelve tasks. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Harness engineering are free. This one comes with the all labs plan.