Figure 16.2: The cheapest eval you can run this afternoon
Figure 16.2 The cheapest eval you can run this afternoon. Give the agent the same task three
times, read all three outputs, and ask of each only “would I accept this?” Here runs 1
and 2 land on the same shape of answer while run 3 diverges (in accent); the spread you
read by hand is the reliability envelope of the previous figure, measured with your own
eyes. Repeated trials, a human grader, an acceptance criterion—the whole discipline
in miniature.
