Evaluating prompts: datasets, grading and regression

How you know a prompt change was an improvement rather than a vibe.

5 min read · Lesson 11 of 12 in this domain

Prompt changes feel obviously better far more often than they are, because you test them on the cases you were just thinking about. The discipline that fixes this is ordinary software practice applied to prompts: a held-out test dataset that includes the awkward cases, a grading method that is consistent enough to compare runs, and regression testing so a fix for one failure mode does not quietly break three others. Grading can be exact-match for extraction, programmatic checks for structure, or a model-based judge for open-ended output — but a judge needs its own explicit criteria, or you have simply moved the vagueness one level down.

• Build a test set that over-samples edge cases; the happy path is not where prompts fail.
• Fix the grading method before you start tuning, so results across runs are comparable.
• Prefer programmatic checks where possible — schema validity, field presence, arithmetic — and reserve model-based judging for genuinely open output.
• A model judge needs explicit, gradeable criteria for the same reason a rubric does: vague criteria produce noisy scores.
• Re-run the whole set on every change. Prompt edits have non-local effects, so a targeted fix routinely regresses something else.
• Segment results rather than reading one aggregate — an average hides subgroup collapse exactly as it does in extraction accuracy.

Source: Claude Docs — Prompt engineering · Independent study aid, not affiliated with or endorsed by Anthropic.