Skip to Content

LLM-as-a-Judge Evaluation

When to use

Use a stronger LLM to grade the output of another LLM — scalable eval when human evaluation is too slow.

Analogy

Hiring a senior professor to grade an exam based on a specific answer key.

Data-flow diagram

   judge_prompt: Rate response 1-5 for accuracy and tone, plus a one-line justification.
   Position bias mitigation: randomise order of options A and B in pairwise eval.

Deep explanation

LLM-as-judge (Zheng 2023) is dominant in modern evaluation because it scales where human eval cannot. Build a clear rubric; ask for a score plus a one-line justification. Always verify with a small human-eval double-check to catch drift in judge quality. Mitigate position bias by randomising ordering; mitigate verbosity bias by normalising length.

Examples

Example 1

def judge(question, response, model='gpt-4o'):
    rubric = ('Rate the assistant response on 1-5 for accuracy and 1-5 for tone.\n'
              'Reply JSON: {"accuracy": <int>, "tone": <int>, "why": "..."}.')
    out = openai.chat.completions.create(
        model=model, response_format={'type':'json_object'},
        messages=[{'role':'system','content': rubric},
                  {'role':'user','content': f'Q: {question}\nA: {response}'}])
    import json; return json.loads(out.choices[0].message.content)

A clear rubric plus structured JSON output minimises judge drift.

Example 2

import random
def pairwise(q, a, b, model='gpt-4o'):
    order = [('A', a), ('B', b)] if random.random() < 0.5 else [('B', b), ('A', a)]
    prompt = (f'Pick which response is better for: {q}\n'
              f'Label1: {order[0][1]}\nLabel2: {order[1][1]}\nAnswer with the label only.')
    out = openai.chat.completions.create(model=model,
        messages=[{'role':'user','content':prompt}])
    return order[0][0] if out.choices[0].message.content.startswith(order[0][0]) else order[1][0]

Pairwise eval with order randomisation removes most position bias.

Example 3

def inter_rater_subset(judge, human):
    return sum(1 for j, h in zip(judge, human) if abs(j - h) <= 1) / len(human)
print('agreement rate:', inter_rater_subset([4,2,5,1], [4,3,5,2]))

Periodically re-check judge agreement with human eval; dump comparisons under 0.6 agreement into a separate review queue.

Common mistake

Letting the same model judge its own output — a self-enhancement bias inflates scores. Use a stronger or different model as judge.

Key takeaway

LLM-as-judge is a scalable first pass; keep a small human gold set to monitor agreement. Use random ordering, structured outputs and per-task rubrics.

Production Failure Playbook

Failure scenario 1: judge-self-enhancement

Failure scenario 2: judge-drift-no-rubric