How model graders are biased

Evaluation, lesson 7

Position, length and self-preference, and how to correct for them.

"Models as graders": how a model judge marks answers, one at a time or two side by side, and what a win rate is.

Part 1. The first answer gets an edge

You ask a judge to compare answers from bot A and bot B on 100 questions, with A's answer shown first. A wins 61. Then you run the same 100 pairs again, with B's answer shown first. Now A wins only 41.

Nothing about the answers changed. Only the order did, and it moved the result by 20 points. Judges (like people) tend to favour whichever option sits in a particular slot, often the first.

Your turn

A wins 61 of 100 when shown first and 41 of 100 when shown second. What's the fairest estimate of how often A is really better?

Each order pushes the result one way by about the same amount, so averaging the two cancels most of the push. The honest reading is close to a tie. Any single order would have told a confident story that was mostly the slot.

Show the answer

B. Each order pushes the result one way by about the same amount, so averaging the two cancels most of the push. The honest reading is close to a tie. Any single order would have told a confident story that was mostly the slot.

Position bias

A judge's habit of favouring an answer because of where it appears, not what it says. The standard fix is to judge every pair in both orders and count a pair whose winner flips with the order as a tie.

This is measured, not folklore. Wang and colleagues (2023) used ChatGPT to judge answers from ChatGPT and from Vicuna-13B, a much smaller model. By changing nothing but the order the answers were shown in, they could make Vicuna win on 66 of 80 questions.

Part 2. Longer looks better

Keep going with Pro

You've done the first part. Pro opens the rest of this lesson, and every other one.

Still to come

Longer looks better
Judges like their own family
Checking a judge against people

See Pro plans I have Pro: sign in

Next lessonPeople as gradersWhen to use human raters, and how far to trust their agreement. Start lesson 8

Evaluation overviewSpotted a mistake? Tell us.

07 · How model graders are biased