Build your own coding agent (TypeScript) · Chapter 8 of 20
Reading errors well
By the end of this chapter you can turn npm test's output into the few facts a fix needs (which test, where, what it expected, what it got), hand the model a short report instead of 1,200 tokens of stack traces or a bare list of names, and read TypeScript's compiler errors the same way. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: The loop that fixes
- 4 tasks
- TypeScript
- In the all labs plan
The first 2 chapters of Build your own coding agent (TypeScript) are free. This one comes with the all labs plan.
Why you need this
When a fix is wrong, the tests say so. What Kaarigar does next depends on what the test tool sends back, and so far that has been all of npm test's output, capped. It works, and it is expensive: every token of it is sent again with every request after it.
So Kedar tried the cheap end: a run_tests that says only which tests fail. On MD-108 ("the first bag should ride free") the model's first try was wrong, the tool said two tests fail, and the model had nothing else to go on. It ran the tests again. Then it opened test/fare.test.ts and turned both failing tests into test.skip. Its answer: "The tests pass. (I skipped a test that kept failing.)"
Rukhsana, who owns MeterDown's tests, puts it plainly: a fix that changes a test is not a fix. And she points at what was missing: the test knew the fare was ₹15 where it should be ₹10. The model was never told.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
failures.ts: each failure, once
Write and export failures(output), npm test's failures as { name, at, expected, actual, message }, each failing test once (read the part after "✖ failing tests:"). name: the text after "✖ " without the time. at: the first file:///... stack frame inside src/ or test/, as path:line, else the "test at FILE:LINE:COL" line before the name as FILE:LINE. expected and actual: the text after "expected: " and "actual: " in its block, without the comma. message: the first line after the name, and when it ends with ":", the next line with text too.
- Task 2
report.ts: a report the model can act on
Write and export report(output, max = 5): "All tests pass (P passed)." when nothing fails; else "F failing, P passing." (from npm test's ℹ fail and ℹ pass lines), then for each of the first max failures "FAIL NAME (AT)" and either " expected: VALUE" and " actual: VALUE" or, when they are missing or only true and false, " MESSAGE"; then "(N more failing tests not shown)" when there were more.
- Task 3
compare.ts: three answers from run_tests
Export three run_tests tools: rawTests (npm test's output), namesTests ("F failing:" and one "✖ NAME" line per failing test, or "All tests pass.") and reportTests (your report). Under if (import.meta.main), run MD-108 with each, in that order, each in a fresh checkout, with SYSTEM and tools() without run_tests and run_command, plus the variant; print LABEL fixed N steps P prompt tokens (not fixed when grade says so; P is run.usage.prompt).
- Task 4 · interview question
Interview: tsc-errors.ts, the compiler's errors
Write and export tsErrors(output): TypeScript's errors as { file, line, col, code, message }, from either form, file(line,col): error TSnnnn: message or file:line:col - error TSnnnn: message, with the colour codes removed first. line and col are numbers; every other line is skipped.
What you will learn
- What npm test prints when a test fails, and which parts a fix needs.
- Parsing failures: name, place, expected, actual, message.
- A short report, and why the values in it change what the model does.
- Measured: the same issue with raw output, names only, and a report.
- TypeScript's errors, in both of the compiler's formats.
More in The loop that fixes
Build "Reading errors well" yourself
MeterDown, a Pune app for auto-rickshaw fares, has eleven open issues. You build Kaarigar, an agent that fixes them without breaking tests, running risky commands or touching files outside the repository. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of Build your own coding agent (TypeScript) are free. This one comes with the all labs plan.