Harness engineering (TypeScript) · Chapter 10 of 23
Loading what is needed
By the end of this chapter you can say, with numbers, why a harness does not paste the whole repository into the prompt, build a repository map (every exported declaration's signature, in about 650 tokens), measure a map against search and against both on T01, T03 and T10, and keep a map from walking into folders that are not the project's code. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Context
- 4 tasks
- TypeScript
- In the all labs plan
The first 2 chapters of Harness engineering (TypeScript) are free. This one comes with the all labs plan.
Why you need this
Farah has a question that sounds like a shortcut. The agent spends its first steps finding code: a search, a read from the hit, sometimes a wrong file first. Hisaab's repository is only 24 files. Why not give the model all of it at the start, and let it skip the finding?
Chapter 9 showed one side of the answer: a project file that grew to 20,897 tokens killed the run on its first request. Pasting the repository is the same mistake on a larger scale, and it hides a second one. The harness you have finds code on demand, one search and one page at a time, and keeps the window for what the task needs. This chapter measures the difference, and the useful middle: a map that says what is where without the code itself.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
stuff-it.ts: paste the whole repository in
Farah asks why the agent does not get every file at the start. In stuff-it.ts, make a T01 checkout and count the tokens (o200k_base) of the text of every file of it (files(root) from ./agent.ts), each file on its own. Print one line per top-level folder in sorted order, then the files at the root, then the total:
docs/: N tokens
src/: N tokens
test/: N tokens
root files: N tokens
everything: N tokens
Then run runAgent on the checkout with T01's issue and a system prompt of SYSTEM followed by every file (a blank line, # PATH, a new line and its text), and print the run's answer as lab-chat: ANSWER.
- Task 2
repo-map.ts: a repository map from the exported declarations
Kabir wants the model told what is where, not handed the code. In repo-map.ts, export repoMap(root, paths): for each file in paths, sorted by path, and each of its lines in order that starts an exported declaration, one line: PATH:LINE: function NAME(ARGS): RET for export function (async function for an async one), PATH:LINE: const NAME: TYPE for export const (no : TYPE without an annotation), and PATH:LINE: type NAME, interface NAME or class NAME. PATH is relative to root, LINE counts from 1, the signature is as written without export and without the body's opening {. Lines that do not start with export are left out. Return the lines joined with new lines. Also export srcFiles(root), the .ts files of root's src/.
Under if (import.meta.main), print the map of a clean checkout's src/*.ts, then N lines, N tokens.
- Task 3
map-vs-search.ts: a map, search, or both
Nandini wants the three ways compared. In map-vs-search.ts, for T01, then T03, then T10, run three harnesses in this order, each on a fresh checkout: search (SYSTEM and TOOLS), map (SYSTEM, a blank line, # Repository map, a new line and repoMap of the checkout's src/*.ts, with toolsWithout('search_code')), and map+search (that prompt with TOOLS). Grade each and print:
T01 map passed 4 steps 7000 tokens
(the task, the harness, passed or failed, result.steps and result.promptTokens, two spaces apart): nine lines.
- Task 4 · interview question
Interview: map-walk.ts, a map that walks too far
Amit's mapFiles gives repoMap every .ts file under the checkout, and with packages installed in node_modules the map is bigger than the window. In map-walk.ts, rewrite mapFiles(root): every .ts file under root, sorted, except the ones under a hidden folder (a name starting with a dot, like .git), node_modules, docs/ or test/. Every other folder counts.
Under if (import.meta.main), twice (Amit's walk, exported as amitsMapFiles, then yours): make a T03 checkout, copy the lab's zod package (join(import.meta.dirname, '..', '..', 'node_modules', 'zod')) into it as node_modules/zod, map the walk's files, run T03 with SYSTEM, a blank line, # Repository map, a new line and the map, and print Amit's walk: N files, N tokens, T03 failed (error, 1 steps), then the same for mapFiles (the files, the map's tokens, the grade, result.stop and result.steps).
What you will learn
- What the whole repository costs, folder by folder, against the window.
- A repository map: every exported name, its signature and its line, read from the declarations.
- Map, search, or both, measured on T01, T03 and T10: steps, tokens and grades.
- What belongs on a map, and what a careless walk drags in.
More in Context
Build "Loading what is needed" yourself
Hisaab, a Lucknow invoicing startup, wants a coding agent to clear its bug list. You build its harness in TypeScript and score it on twelve tasks. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of Harness engineering (TypeScript) are free. This one comes with the all labs plan.