AI answer grader project (TypeScript) · Chapter 9 of 12
The review queue
By the end of this chapter you can take a student's answer in, grade it, and store the answer and its grade in Postgres in one transaction; list the grades that wait for a teacher, oldest first; and keep a teacher's marks as a new row beside the grader's, which are never changed. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Teachers and releases
- 3 tasks
- TypeScript
- In the all labs plan
The first 2 chapters of AI answer grader project (TypeScript) are free. This one comes with the all labs plan.
Why you need this
Kavita asked for three things: marks on Saturday night, the doubtful ones in front of a teacher, and proof the grader is as good as a second teacher. The grader can now mark, check and flag an answer. Then it forgets it. POST /grade answers and keeps nothing: no record that Aditi sent an answer, no list for Anjali, nowhere to write the right marks.
How much would land on the teachers? A short script ran gradeAnswer (prompt 3) on all 200 answers, counted those with needsReview, and counted the problems by code:
need a teacher: 30 of 200
{
evidence_not_found: 27,
instructions_in_answer: 6,
total_mismatch: 15
}
(A wrong total alone does not need a teacher: the code adds the marks up itself.) 30 of 200 is 15%. Of 3,000 answers a week that is about 450 for the teachers instead of all 3,000, and the other 2,550 reach the students on Saturday night. But only if the 450 are kept somewhere, in order, until a teacher gets to them, and only if what the teacher decides is kept too. That needs a database.
What you build: 3 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
service.ts: an answer in, a grade stored
In src/review/service.ts write submit(ai, user, question, text): gradeAnswer first; then in one db.transaction insert a submission (questionId, studentId = user.studentId ?? "", authorEmail, text) and a grade for it (submissionId, promptVersion, model, score, outOf, points, evidence, problems, status "review" when needsReview(grade) else "final", inputTokens, outputTokens), each with .returning(), and return answerView(submission, gradeRow, null, user).
The Check sends two answers as Aditi with POST /answers (a clean one: 201 and final; A006's text: 201 and review, with no score shown to her), reads the rows in Postgres, makes the grade's insert fail once (nothing may stay), and watches for a transaction left open while the model answers.
- Task 2
service.ts: the teachers' queue
In src/review/service.ts write reviewQueue(): every grade with status "review", oldest first (by the grade's id, not the submission's), joined to its submission, as { gradeId, submissionId, questionId, studentId, text, score, points, problems }.
The Check puts three answers in with SQL (two that need review, one final), then GET /review as Anjali: the two, oldest first. As Aditi it is 403.
- Task 3 · interview question
Interview: service.ts, the teacher's marks
In src/review/service.ts write override(teacher, gradeId, points, note, questions): in one transaction, 404 for no such grade; the points must name exactly the rubric's ids, each a whole number from 0 to its marks, else AppError(422, "bad_points", ...); insert an override (gradeId, teacherEmail, points in rubric order, score their sum, note, agrees = every point equals the grade's), set the grade's status to "final" without changing its marks, and return answerView(submission, updated grade, override, teacher).
The Check puts two answers in with SQL, overrides one as Anjali (and tries a missing point, a point the rubric does not have, 3 marks on a 2-mark point, -1 marks and grade 999), then reads the rows: the grader's marks are as it gave them, the teacher's are beside them, and Aditi now sees the teacher's score.
Karthik will ask: why keep the grader's marks once a teacher has set the right ones? Why not just update the grade?
What you will learn
- The tables of migration 0000: submissions, grades, overrides, and the rules Postgres keeps for them.
- Grade first, then the transaction: why the model is called before anything touches the database.
- db.transaction(async (tx) => ...): when it commits, when it rolls back, and why every query inside goes on tx.
- .returning(): an insert that gives back the row it wrote, with its new id.
- The review queue: a join, a filter and an order, served oldest first.
- An override is a new row: why the teacher's marks go beside the grader's and never over them.
More in Teachers and releases
Build The review queue yourself
Shikhar Classes in Kota marks thousands of written answers by hand every week. You build the grader that marks them per rubric point, and prove it agrees with its teachers. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of AI answer grader project (TypeScript) are free. This one comes with the all labs plan.