The Golden Set

Every Product Owner has said some version of this in a Sprint Review: “That is not quite what I meant by good.”

For a conventional feature, that sentence starts a productive conversation. The gap is usually visible on screen, and closing it is a matter of adjusting something specific. For an AI feature, the same sentence starts a much worse conversation, because there is no shared reference for what good would have looked like. The Product Owner has an impression. The Developers have a working system. Neither can point at anything.

A golden set is what you point at.

What It Is

A golden set is a curated collection of real inputs, each with an agreed expectation of what a good output looks like, used to score the feature's behaviour repeatedly over time.

The expectation does not have to be a single perfect answer, and for most interesting features it cannot be. It might be:

• A reference output someone wrote by hand
• A list of facts that must appear, and facts that must not
• A rubric describing the properties a good answer has
• A set of answers that would all be acceptable

If you have worked with acceptance criteria and examples, this will feel familiar — it is close in spirit to specification by example. The difference is that the examples are not illustrations of the rule. For a probabilistic feature, the examples are the specification, because there is no rule that can be written down more precisely than the examples themselves.

Building the First One

The most common reason teams never build a golden set is that they imagine it as a large, formal dataset requiring specialist skills. It is not. The first one is a spreadsheet, and it takes an afternoon.

Step 1: Take twenty to fifty real cases

Real, not invented. Pull them from actual usage, actual tickets, actual documents, actual queries. Invented cases are systematically too clean — they lack the ambiguity, the missing fields, the contradictory information, and the odd formatting that real inputs have, which is exactly where the feature will fail.

Twenty is enough to start. A small set that exists beats a large one that is still being designed.

Step 2: Include the cases you are afraid of

Most golden sets are built from typical cases, which means they measure the feature on the inputs least likely to cause harm. Deliberately include:

• The ambiguous case where two answers are defensible
• The case where the correct answer is “I do not know” or “insufficient information”
• The case where the input contains something that must never appear in output
• The case that went badly the last time a human handled it

These cases are worth several times a typical one, because they are where the difference between a 90% feature and a 97% feature actually lives.

Step 3: Write down what good means, case by case

This is the step where the Product Owner earns their place in the room, and it is usually harder than expected. Writing the expectation for thirty real cases surfaces disagreements about the product that had been comfortably vague until now. Two stakeholders will turn out to want different things. A rule everyone thought was settled will turn out to have three exceptions.

That discovery is not an obstacle to building the golden set. It is most of the value of building it. Those disagreements were going to surface eventually — the choice is whether they surface in a Refinement session or in production.

Step 4: Separate what must never fail

Split the set into a general portion and a critical portion. The general portion carries a threshold that reflects an acceptable level of imperfection — 90%, say. The critical portion carries 100%, and it holds the cases where a single failure is genuinely unacceptable: personal data leaking into output, a fabricated reference, a safety-relevant claim.

This split is what makes the threshold conversation tractable. Without it, the Product Owner is asked to pick one number that covers both “occasionally clumsy phrasing” and “occasionally exposes a customer's email address,” and no single number can honestly do that.

Sizing It as Real Work

Here is the failure that repeats across teams, in almost identical form.

An item enters the Sprint. The Developers build the feature. Nobody sized the golden set, because it did not feel like feature work. At Sprint Review, the feature demonstrably runs — and cannot be shown to meet its acceptance criteria, because the thing that would demonstrate this does not exist. The item is not Done, and the reason is not visible in any of the work that was tracked.

Two practices prevent this.

Build the golden set before the feature, not after. Written first, it functions as the specification the Developers build toward. Written afterwards, it tends to get shaped — usually unconsciously — around what the system already does, which is how a set that reports 95% coexists with users who are unhappy.

Make it visible in the Sprint Backlog. Whether it is its own Product Backlog item or a task within one matters less than that the effort is forecast rather than absorbed. A team doing this for the first time should expect it to take longer than they think, mostly because of the disagreements in step 3.

Growing It From Failure

The golden set that matters after a year is not the one designed at the start. It is the one built from everything that actually went wrong.

The habit to establish is simple and costs almost nothing: every production failure becomes a new case. Someone complains that a summary invented a detail. That input goes into the set, with the expectation written down. Now the same failure cannot silently return.

This is the same instinct as writing a failing test before fixing a bug, and framing it that way is how you get experienced Developers to adopt it without a debate about process. It is also the mechanism that makes the set progressively harder over time, which is what you want — a golden set that always reports 98% has stopped measuring anything.

Three Failures Worth Avoiding

Building it from generated cases. If a model produces the inputs and a model produces the expected outputs, the set measures whether the system agrees with itself. This is tempting because it is fast, and it is worse than having no set at all — because the resulting number carries false authority.

Letting the judge grade uncalibrated. If a second model is scoring outputs against your rubric, someone has to check that its scores track human judgement before anyone believes them. Grade fifty cases by hand, compare, and measure the agreement. This is covered further in acceptance criteria for non-deterministic AI features.

Treating the threshold as permanent. The number that was right at launch is usually wrong six months later, in either direction. It belongs on the agenda when the feature's usage changes materially, and the Product Owner is the one who moves it.

What a Product Owner Should Take Away

You already own the definition of value. For a conventional feature, that ownership is expressed through the Product Backlog, its ordering, and the conversations around each item. For an AI feature, a large part of it is expressed through the golden set, because that is where “good” stops being a word and becomes something a team can act on.

Refusing that work does not mean the definition goes unwritten. It means it gets written by whoever is closest to the implementation, at the moment they need an answer, without anyone recording that a decision was made. That is one of the more reliable ways a team accumulates verification debt — work formally accepted and substantively unexamined.