LLM-as-a-Judge

What LLM-as-a-Judge Means

LLM-as-a-judge is an evaluation method in which a capable language model reads one or more candidate outputs and produces a quality signal: a numeric score, a category label, or a preference between two answers. The judge model stands in for a human annotator. It receives an instruction (a rubric or grading prompt), the input that produced the candidate, and the candidate text, then returns its verdict. The motivation is practical. Human evaluation of open-ended generation is slow, expensive, and hard to scale across thousands of examples and frequent model revisions, while reference-based metrics like BLEU and ROUGE correlate poorly with human judgment on tasks that reward creativity and reasoning.

The approach was formalized and popularized by the 2023 paper "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena" by Zheng et al., which reported that strong judges such as GPT-4 can match both controlled and crowdsourced human preferences, achieving over 80% agreement, roughly the level of agreement between two independent human annotators. That result is the empirical backbone for treating an LLM judge as a usable proxy for human preference, with the caveat that agreement is not perfect and the judge introduces systematic errors of its own.

An LLM judge is not a calibrated measurement instrument. It is a generator conditioned to emit a verdict, so its output reflects the rubric wording, the ordering of candidates, the length of responses, and the judge model's own training distribution. Understanding the scoring modes and the known failure modes is what separates a reliable evaluation harness from a misleading one.

• A judge model converts free-text outputs into a comparable signal: a score, a label, or a preference.
• It substitutes for human raters when human evaluation is too slow or costly to scale.
• Zheng et al. (2023) reported strong judges reaching over 80% agreement with human preference.
• The judge is a conditioned generator, not a calibrated meter, so its verdicts inherit model biases.

Pointwise, Pairwise, and Reference-Based Scoring

There are three common scoring modes. Pointwise (also called single-answer or direct) scoring shows the judge one candidate and asks for an absolute rating, often on a 1 to 10 scale or against a checklist rubric. It is cheap because each candidate is judged once, but absolute scores from an LLM are weakly calibrated and tend to drift between runs, so pointwise numbers are most useful in aggregate or relative to a fixed baseline rather than as standalone quality measures.

Pairwise comparison shows the judge two candidates, A and B, for the same input and asks which is better (with ties allowed). Humans and models both find relative judgments easier and more consistent than absolute ones, so pairwise judging usually agrees with human preference more reliably. Its cost is that the number of comparisons grows with the number of systems, and raw win counts must be aggregated into rankings, commonly with the Bradley-Terry model that underlies the Chatbot Arena leaderboard.

Reference-based scoring gives the judge a gold answer alongside the candidate and asks how well the candidate matches the intended meaning or required facts. This grounds the verdict and is well suited to tasks with a knowable correct answer, such as closed-form question answering or retrieval-augmented generation where the supporting passage is available. Reference-free variants, including the G-Eval framework of Liu et al. (2023) which pairs a judge with chain-of-thought rubric steps, are used when no gold answer exists, as in summarization quality or dialogue coherence.

• Pointwise: judge one answer for an absolute score; cheap but weakly calibrated and run-to-run noisy.
• Pairwise: judge A versus B; more consistent with human preference but scales with comparison count.
• Reference-based: compare a candidate against a gold answer; best where a correct answer is knowable.
• Reference-free rubric methods like G-Eval handle open-ended tasks with no single right answer.

Known Biases and Failure Modes

LLM judges exhibit systematic biases that a naive harness will silently absorb. Position bias is the tendency to favor a candidate based on where it appears in the prompt, for example preferring whichever answer is shown first in a pairwise comparison regardless of content. Zheng et al. (2023) document this directly and recommend swapping the order of A and B and only counting a win when the judge prefers the same answer in both orderings, which both detects and neutralizes the effect.

Verbosity bias is the tendency to score longer answers higher even when the extra length adds no correctness, which can reward padding and penalize concise correct responses. Self-enhancement bias (also called self-preference) is the tendency of a judge to favor its own answers, rating outputs from itself or its own model family more highly than a blind human would. It is especially dangerous when the same model is both a contestant and the judge. The 2023 paper identifies all three as named limitations of the LLM-as-a-judge setup.

Beyond these, judges can be inconsistent across runs because of sampling, sensitive to small rubric rewordings, and vulnerable to prompt-injection-style manipulation where a candidate output contains text instructing the judge to rate it highly. Practical evaluation pipelines counter these with order swapping, length controls, using a judge from a different family than the systems under test, fixing low temperature for repeatability, and periodically validating the judge against a human-labeled gold set to confirm agreement has not drifted.

• Position bias: favoring an answer by its slot; mitigated by swapping order and requiring a consistent win.
• Verbosity bias: rewarding length over correctness, penalizing concise but accurate answers.
• Self-enhancement bias: a judge preferring its own or its family's outputs, unsafe when judge equals contestant.
• Run-to-run inconsistency and prompt-injection make a human-labeled gold check essential for trust.

A Minimal Pairwise Judge

A pairwise judge is, at its core, a single structured prompt: a rubric, the shared input, the two candidates labeled A and B, and a request for a verdict in a parseable format. The example below issues one comparison through the OpenAI Python SDK. The judge is asked to return only a token so the result is easy to parse, and the prompt instructs it to ignore length and ordering, a partial guard against verbosity and position bias.

Two practices turn this skeleton into a defensible measurement. First, run every pair twice with A and B swapped and count a decisive win only when the verdict is stable across both orderings; ties or flips become ties. Second, use a judge model from a different family than the systems being graded to avoid self-enhancement bias. Aggregating the resulting stable win counts with the Bradley-Terry model yields a ranking with confidence intervals rather than a single fragile number.

from openai import OpenAI

client = OpenAI()

JUDGE = (
    "You are an impartial judge. Compare answers A and B to the same "
    "question. Ignore their length and their order. Reply with exactly "
    "one token: A, B, or tie."
)

def judge_pair(question, answer_a, answer_b, model):
    prompt = (
        f"Question:\n{question}\n\n"
        f"Answer A:\n{answer_a}\n\n"
        f"Answer B:\n{answer_b}\n\n"
        "Which answer is better?"
    )
    resp = client.chat.completions.create(
        model=model,
        temperature=0,
        messages=[
            {"role": "system", "content": JUDGE},
            {"role": "user", "content": prompt},
        ],
    )
    return resp.choices[0].message.content.strip()

# Swap order and keep only a consistent verdict to blunt position bias.
def stable_verdict(question, a, b, model):
    first = judge_pair(question, a, b, model)
    second = judge_pair(question, b, a, model)
    if first == "A" and second == "B":
        return "A"
    if first == "B" and second == "A":
        return "B"
    return "tie"

• The judge is one structured prompt: rubric, shared input, candidates A and B, and a parseable verdict.
• Constrain the output to a single token (A, B, or tie) to make parsing and aggregation reliable.
• Run each pair in both orderings and keep only verdicts that stay consistent across the swap.
• Choose a judge outside the contestants' model family to reduce self-preference.

When to Trust an LLM Judge

An LLM judge is appropriate when human evaluation cannot keep pace with iteration: regression testing across model versions, ranking many systems, or scoring outputs on subjective axes like helpfulness and tone where exact-match metrics fail. It is a proxy, so the right question is not whether it is perfect but whether its agreement with human labels on a representative gold set is high enough for the decision at hand. A judge used to pick between two release candidates needs less precision than one whose scores gate a model from shipping.

The judge is a poor fit where correctness is objectively checkable by cheaper means. Unit tests, exact-match graders, numeric tolerance checks, and schema validators are more reliable and far cheaper than a model for code that either compiles or does not, math that has a single answer, or JSON that either parses or does not. Reserve the LLM judge for the genuinely open-ended slice of the evaluation, and anchor even that with periodic human spot checks.

Treating the judge as part of the system under measurement, with its own validation, makes it trustworthy. That means logging judge prompts and verdicts, tracking judge-human agreement over time, pinning the judge model and decoding settings for reproducibility, and re-checking agreement whenever the judge model is upgraded, since a new judge version can shift verdicts even on unchanged candidates.

• Best for subjective, open-ended, high-volume evaluation where human raters cannot keep up.
• Prefer deterministic graders (tests, exact match, schema checks) wherever correctness is objectively checkable.
• Validate the judge against a human-labeled gold set sized for the stakes of the decision.
• Pin the judge model and decoding settings, and re-verify agreement after any judge upgrade.