Use this when

You will change an AI workflow over time, tweak a prompt, swap a model, adjust a rule, and need to be sure each change does not quietly make things worse.

The pattern

GOLDEN TEST SET
real cases + agreed correct answers, kept stable
        ↓
run it after EVERY change to the workflow
        ↓
results worse than before? → investigate before shipping

A fixed, curated set of real cases with known good answers, rerun whenever anything changes.

Why it works

Changes to AI workflows have surprising side effects. Fixing one case can break another. A golden set catches this: because the cases and their ground truth stay stable, any drop in results points straight at your latest change. It turns "I think that is better" into "it improved these cases and broke none."

This is how you catch a regression, a change that quietly reduces quality, before your users do.

How to apply it

  1. Build it from real cases with agreed answers. See how to create a test set from real work.
  2. Keep it stable, so results are comparable across changes.
  3. Rerun it after every change and compare to the previous results.
  4. Add new cases over time, especially real failures found in operation, so the set grows with what you learn.

Good fit

  • Any workflow you expect to maintain and improve over months.
  • Teams making frequent small changes to prompts, rules or models.

Poor fit

  • A one-off task you will never change, where the ongoing upkeep is not worth it.

What this does not solve

A golden set catches regressions on known cases; it cannot cover every future input. It is a safety net for change, not a proof of correctness. Pair it with monitoring, so new failures in the wild become new cases in the set.