Home / Glossary / three-set discipline

three-set discipline

Developing an LLM classifier without fooling yourself, using three disjoint sets: the demonstrations packaged into the skill, the development set you iterate against, and a held-out test set consulted rarely and retired when overused. The training set could be dispensed with; the discipline that grew up around it could not (Chapter 26).