In 30 seconds

A test set is a collection of real cases with known good answers. It lets you measure how the AI actually performs, instead of trusting a demo. Building one from real work is the single most useful testing step.

Why this comes up

A demo shows the AI handling cases someone chose. It tells you almost nothing about how it handles the messy reality of real work. A test set fixes that: real cases, agreed answers, measured results. It turns "it seemed to work" into "it was right on 92 of 100 real cases, and here is where it failed."

Gather real cases

Pull cases from actual past work, not invented ones. Real cases carry the quirks, formats and exceptions that matter. Include a mix:

  • Common, normal cases, the bulk of the work.
  • Edge cases: unusual formats, missing information, rare categories. See how to test edge cases.
  • Prohibited cases: inputs where the AI should refuse or escalate, not act.
  • Cases with conflicting or incomplete information.

A test set of only easy cases will make a weak workflow look strong.

Agree the correct answer for each

For each case, record the correct result, the ground truth, agreed by someone who knows the work. This is the standard the AI's output is compared against. Where reasonable people might disagree, note that; it tells you the task itself has genuine ambiguity.

Keep it and reuse it

Save the test set and run the workflow against it whenever you change something. This turns it into a golden test set: a stable benchmark that catches when a change quietly makes things worse. Add new cases over time, especially real failures you discover in operation.

A small example

A team evaluating an invoice extractor built a set of 80 real past invoices, including a few handwritten ones, some with missing fields, and two that were not invoices at all. Measuring against agreed answers showed the extractor was strong on clean invoices and weak on handwriting, exactly the detail a demo had hidden.

What this does not solve

A test set measures performance on the cases it contains; it cannot cover every future case. New, unseen situations will still occur. Keep adding real cases, and pair the test set with monitoring after launch. A test set reduces surprises; it does not remove them.

Good enough to move on when

  • The test set is built from real cases.
  • It includes normal, edge and prohibited cases.
  • Each case has an agreed correct answer.
  • It is saved and rerun whenever the workflow changes.