LLM as a judge is an evaluation method where one language model scores another system’s output against a rubric. Teams use it for qualities that code cannot check, such as helpfulness, tone, or whether an answer is supported by its sources. LangChain describes it as scoring “against defined criteria”, and the defining paper, Zheng et al., presented it as a scalable way to approximate human preferences, which are expensive to collect.

How a judge works

A judge needs three things: the output to evaluate, evidence to evaluate it against, and a rubric. LangChain lists these as the core inputs, and says the judge returns a pass/fail, a category, or a number, often with a short explanation. The judge only sees what you give it, so it can assess tool use only if the tool calls are in its input.

Zheng et al. describe three formats:

FormatWhat the judge doesTrade-off
Pairwise comparisonSees a question and two answers, picks the better one or declares a tiePairs grow quadratically as you add candidates
Single answer gradingAssigns a score to one answerScalable, but absolute scores fluctuate more than relative pairwise results
Reference-guided gradingGets a reference solution to compare againstMitigates a judge being misled when grading math and reasoning answers

LangChain uses the same split in practice: individual scoring for monitoring and regression checks, pairwise for comparing prompts, models or agent versions, and reference-based for RAG and correctness checks. It also says to keep the judge and rubric fixed when comparing agent versions, otherwise a scoring change can look like an improvement.

When to use one

LangChain recommends a judge when the criterion is subjective or context-dependent, when many wordings could be correct, or when volume makes human review of every response impractical. For fixed requirements such as valid JSON or required fields, it suggests a code evaluator instead, and it says human review is necessary for high-stakes decisions and for the labeled examples that validate the judge.

Known biases

Zheng et al. named and tested four weaknesses. The numbers below come from 2023 models, are there to show the size of the effect, and may not hold for today’s judges.

BiasWhat happensWhat the 2023 study found
PositionThe judge favors an answer because of where it sitsWith two near-identical answers, only GPT-4 gave the same verdict after a swap in more than 60% of cases, and most judges favored the first position
VerbosityLonger answers win even when they are no betterWhen answers were padded with a repeated list, Claude-v1 and GPT-3.5 were fooled 91.3% of the time and GPT-4 8.7%
Self-enhancementThe judge favors its own model’s answersGPT-4 gave itself a 10% higher win rate and Claude-v1 25%, but GPT-3.5 did not, and the authors said the data could not settle it
Weak grading of math and reasoningThe judge is misled by the answers it gradesOn 10 math questions, the default prompt failed 14 of 20 judgments, chain-of-thought 6 of 20, a reference answer 3 of 20

Repeatability is not the same as accuracy. LangChain says to check accuracy and repeatability separately, because a judge can be repeatable and still wrong.

On self-preference, Arize cites Wataoka et al. as attributing it to perplexity rather than recognizing authorship, and advises using a different model for the judge than for the application. LangChain adds that a judge on the same model family may share its blind spots.

How to reduce the biases

  • Swap positions. Zheng et al. suggest calling the judge twice with the order swapped and declaring a win only when an answer is preferred both times, treating the rest as ties. They add that assigning positions randomly can work at large scale.
  • Show examples. The same paper found that three few-shot judging examples raised GPT-4’s consistency in their position test from 65.0% to 77.5%, though they warn higher consistency may not mean higher accuracy and the longer prompts cost 4x more.
  • Give the judge a reference or reasoning step for math and logic tasks, with the caveat in the paper that chain-of-thought still often repeats the graded answer’s mistake.
  • Write explicit criteria. Arize contrasts “score this response from 1 to 5” with “score 1 if the response cites the retrieved context, 0 otherwise”.
  • Prefer small scales. Arize says large numeric scales do worse than small ones or labels, and suggests combining binary sub-criteria for granularity. LangChain makes the same point: if the judge cannot tell a 70 from an 80 repeatably, the scale is too precise. Hamel Husain advises starting with binary pass/fail plus a written critique from a domain expert.

How to check a judge before trusting it

A judge that runs is not a judge that is right. The steps below combine what the sources agree on.

  1. Label a sample by hand. Arize suggests 20 to 100 examples spanning the score range, labeled by people using the same rubric the judge gets.
  2. Hold some out. LangChain says to test revised judges on labeled examples that were not used to tune the prompt.
  3. Check position bias and repeatability. Run pairwise comparisons in both orders, as Zheng et al. suggest, and LangChain says to track accuracy and repeatability separately.
  4. Look at both error types. LangChain says to review false passes and false failures separately, because a false pass lets a bad response through and a false failure adds noise.
  5. Repeat after changes. Husain says that when you change the judge’s model you go through the process again and measure the results. Humans stay in the loop, he writes, because the judge is aligned to a human standard.

The 2023 study reported that GPT-4 agreed with human experts on 85% of non-tie MT-bench comparisons, against 81% between humans. That is one benchmark and 2023 models, so a judge that looks good there is not proven on your data. LangChain warns against relying on one overall agreement number.

For a case where human scoring comes first and an LLM judge only fills in later, see the walkthrough Using an agent to improve another agent.