AI Agents, Engineered

Home / Figures / Chapter 16

Figure 16.3: The admitted circularity of using a model as a judge: the grader and the graded are the same kind of creature—useful once calibrated, and warmly disposed toward its own handwriting

A model's output is graded by a judge model of the same kind against a rubric, and the judge's verdicts are calibrated against human labels before they are trusted.
Figure 16.3 The admitted circularity of using a model as a judge: the grader and the graded are the same kind of creature—useful once calibrated, and warmly disposed toward its own handwriting.