Harness engineering · Chapter 9 of 23
Project files
By the end of this chapter you can find a repository's project file (AGENTS.md) the way coding assistants do, put it in the system prompt, load a folder's own file only when the agent first works in that folder, cap a file that has grown too big for the window, and say, with numbers from the suite, what the project file is worth. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Context
- 4 tasks
- Python
- In the all labs plan
The first 2 chapters of Harness engineering are free. This one comes with the all labs plan.
Why you need this
Chapter 8's prompt knows Hisaab because you typed Hisaab into its config: the language, the folders, the test command. That works for one repository and one engineer who knows it. It does not know the things only Hisaab's team knows, and T04 needs one of them.
T04's issue: gst_amount(2525, 18) gives 454 paise, and GST rounds half up, so it should be 455. Under chapter 8's written prompt, the model finds gst_amount, and its first fix is the one most Python code would use:
edit_file(hisaab/gst.py, "return round(base_paise * rate / 100)"
-> "return int(base_paise * rate / 100 + 0.5)")
run_command(python -m pytest -q) -> 1 failed, 44 passed
E AssertionError: credit notes too: round half away from zero in
integer paise, with money.round_half_up
edit_file(hisaab/gst.py, the first try -> round_half_up(base_paise * rate, 100))
run_command(python -m pytest -q) -> 45 passed
It passes, in 7 steps, because the prompt says to run the tests and the test's message happens to name the right function: int(x + 0.5) rounds a credit note of -2525 paise at 18% to -454, not -455. Under the one-line prompt nothing runs the tests, the first try stays, and T04 fails. Hisaab's rule is "money is integer paise, rounded with money.round_half_up", and Kabir has said it in every code review for two years. It is written down in one place, which the harness never reads: the project file.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
load_project.py: find the project file, put it in the prompt
Kabir wants the harness to read what the repository says about itself. In load_project.py, write project_files(start): every AGENTS.md from the repository's root down to the folder start, outermost first, as a list of paths. The root is the nearest folder, from start upward, with a .git in it; never look above it. No file: an empty list.
Then project_prompt(base, start): the base prompt, then for each file a blank line, # Project file (PATH) with PATH relative to the root (AGENTS.md, hisaab/AGENTS.md), a new line and the file's text. With no file, the base unchanged.
Under if __name__ == "__main__":, run run_agent on a T01 checkout with project_prompt(SYSTEM, root) (SYSTEM from agent.py) and print as the last line T01: passed in N steps (or failed).
- Task 2
nested.py: a folder's own file, loaded on first touch
A folder can have its own AGENTS.md. In nested.py, write a class NestedLoader(root). touch(path), path relative to root: for every folder from just below root down to the path's folder, outermost first, that it has not loaded before, mark it loaded and, if it has an AGENTS.md, take # Project file (PATH), a new line and its text (PATH from the root). Return those blocks joined by a blank line, or None when there is nothing new. The root's own file is already in the prompt (never load it), and a path outside root loads nothing.
on_tool(name, args): for read_file and edit_file, touch(args["path"]); for any other tool, None. agent.py's run_agent takes on_tool=loader.on_tool and adds what it returns to the system message. The check runs T04 with a hisaab/AGENTS.md that has the rounding rule: the rule must arrive after the first read in hisaab/, and the first fix must be right.
- Task 3
t04_both.py: T04 without and with the project file
Nandini wants the project file's worth in numbers. In t04_both.py, run T04 four times, each on a fresh checkout, in this order: the one-line prompt "You are a coding agent. Fix the issue." without and then with the project file (project_prompt of that prompt), then SYSTEM without and then with it. Grade each one, count its test runs (the run_command calls in result.messages whose arguments have pytest in them), and print:
T04 written with passed 5 steps tests run: 1
(the task, one-line or written, without or with, passed or failed, the steps and the test runs, two spaces apart).
- Task 4 · interview question
Interview: size_cap.py, a project file too big for the window
Someone pasted the GST guide into AGENTS.md, and the first request is refused. In size_cap.py, write load_capped(path, cap): the file's text if it is at most cap tokens (o200k_base); otherwise its first whole lines that fit in cap tokens, then a line [AGENTS.md cut: N of M tokens shown; read the file for the rest] (the file's own name, N the tokens of the kept text, M the whole file's). Then capped_prompt(base, start, cap=1500): project_prompt's layout with each file through load_capped.
Under if __name__ == "__main__":, on a fresh T01 checkout whose AGENTS.md has a new line and docs/gst-guide.md appended, print AGENTS.md: N tokens, run run_agent with project_prompt(SYSTEM, root) and print uncapped: STOP GRADE N steps (result.stop, passed or failed, the steps, two spaces apart); then on another such checkout, the same with capped_prompt, as capped: STOP GRADE N steps.
What you will learn
- What a project file is: how a repository tells every agent its layout, commands and rules.
- Finding it the way coding assistants do: from the folder the agent starts in, up to the repository's root.
- Putting it in the prompt, and what it changes on T04 and on the steps of other tasks.
- Nested files, loaded the first time the agent works in their folder.
- A size cap, for a project file that has grown too big.
More in Context
Build "Project files" yourself
Hisaab, a Lucknow invoicing startup, wants a coding agent to clear its bug list. You build its harness and score it on twelve tasks. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of Harness engineering are free. This one comes with the all labs plan.