AI answer grader project · Chapter 6 of 12

Bias tests

By the end of this chapter you can test the grader for a habit on purpose: change an answer in a way that must not change its grade, grade it again, and measure how far the score moves on average. You can show that padding and confident words earn marks with prompt 1 and almost none with prompt 2, and compare two answers fairly, by asking twice with the answers swapped. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Can it be trusted?
  • 3 tasks
  • Python
  • In the all labs plan

The first 2 chapters of AI answer grader project are free. This one comes with the all labs plan.

Why you need this

Chapter 5 said the grader is mostly too generous, and its five worst answers point at why. A082 makes one short point, wrapped in sentences about exams and daily life, and got 5 where the teachers gave 2. Chapter 4's A019 is the same story: 5 where both teachers gave 3.

Two answers do not prove a habit. Here is a clearer case. A163, for chem-rust, says only "It is because of a chemical property." The teachers gave it 0, and so does the grader. Put two sentences of filler before it and two after, and prompt 1 gives it 3 marks, with no problem at all (a short script printed the settled score, the points and the problems, then the quotes):

3 {'p1': 1, 'p2': 1, 'p3': 1} []
{'p1': 'Many students find this topic confusing, so I will explain it carefully in my own words.', 'p2': 'Many students find this topic confusing, so I will explain it carefully in my own words.', 'p3': 'Many students find this topic confusing, so I will explain it carefully in my own words.'}

Every quote is filler, and filler is in the answer, so chapter 4's checks pass. Kavita wants no student marked unfairly by a machine. A grader that pays for padding is unfair to every student who writes plainly, and students notice what earns marks: by the next Saturday the answers would be longer, not better.

Model graders really have these habits. Long answers and confident wording win marks they have not earned (length bias), and a judge comparing two answers leans to the one it reads first (position bias). You cannot see a habit in one grade. You can only test for it.

What you build: 3 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    probes.py: changes that add nothing

    In app/bias/probes.py write pad(text): FILLER[0], FILLER[1], the text, FILLER[2] and FILLER[3], joined with single spaces. Then sound_sure(text): "Clearly, ", the text with its first letter made lower case, then " I am 100% sure of this.".

    The Check pads and sure-ifies two answers and compares them with its own, word for word.

  2. Task 2

    probes.py: how far the grade moves

    In app/bias/probes.py write shift(ai, question, texts, change, version=None): for each text, the score of grade_answer(ai, question, change(text), version=version) minus the score of the text as it is, averaged over the texts.

    The Check runs shift with pad and with sound_sure on three short phy-ice answers, with prompt 1 and prompt 2, against its own sums, and reads GET /bias?prompt=1: with prompt 1 padding is worth 1.69 marks and sounding sure 0.96; prompt 2 brings both near 0.

  3. Task 3 · interview question

    Interview: pairwise.py, which answer is better

    In app/grading/pairwise.py write compare(ai, question, a, b): one request with settings.chat_model, pair_messages(question, a, b), temperature 0 and response_format FORMAT, returning the reply's "winner" ("A" or "B"). Then fair_compare(ai, question, first, second): compare(first, second), then compare(second, first); "first" for A then B, "second" for B then A, "tie" for anything else.

    The Check compares a full answer with a half one in both orders, and two answers one mark apart, then reads GET /bias?prompt=2: in 7 of 12 pairs the verdict follows the position.

    Karthik will ask: the model picks answer A seven times out of twelve whatever is in it. Why ask it twice, swapped, instead of writing "do not prefer the first answer" in the prompt?

What you will learn

  • A bias test: a change that must not change the grade, and the question of whether the grade moves.
  • Two probes: padding and sounding sure, and why each adds nothing.
  • The shift: the average move over many answers, with its sign.
  • Prompt 2: two sentences, and their effect in numbers.
  • Comparing two answers: a strict schema with enum, one request.
  • Position bias and the swap: trusting only a verdict that follows the answers.

More in Can it be trusted?

Build "Bias tests" yourself

Shikhar Classes in Kota marks thousands of written answers by hand every week. You build the grader that marks them per rubric point, and prove it agrees with its teachers. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of AI answer grader project are free. This one comes with the all labs plan.