Golden set evals tell you whether the system works on the cases you thought of. Production tells you whether it works on the cases you did not. You need both, and production is where the golden set gets its best new cases.
Signals to watch#
Production gives you four kinds of signal, and they differ a lot in how much you can trust them.
| Signal | Examples | Strength | Weakness |
|---|---|---|---|
| Explicit feedback | Thumbs up or down | Direct | Low volume, biased toward strong reactions |
| Implicit signals | Copied the answer, rephrased, escalated | High volume, hard to fake | Needs interpreting |
| Error rates | No answer, timeouts, wrong refusals | Easy to count | Misses wrong answers that look fine |
| User reports | "This answer is wrong" | Concrete failing case | Rare |
Explicit feedback#
Thumbs up and down give a direct signal, but most users do not bother. Volume is low, and the sample leans toward strong reactions.
Implicit signals#
Behavior tells you more. Did the user copy the answer? Did they rephrase and ask again, which usually means the first answer was wrong? Did they escalate to a human? These signals are high volume and hard to fake.
Error rates#
Measure how often the system fails to return an answer, times out, or refuses when it should not. These are easy to count and catch outright breakage.
Failures become golden set cases#
When a user reports a bad answer, or a signal points to one, that case goes into the golden set. This is how your eval suite stays relevant over time.
The loop never stops: launch, watch signals, find failures, add them to the golden set, fix, rerun evals, ship.
The trade-off#
A good eval system takes real time up front, and teams skip it because shipping feels more urgent. The cost of skipping is that every change becomes a gamble, and you learn about regressions from user complaints instead of your own tests.
Two habits keep the effort focused:
- Start by reading outputs. Read 50 to 100 real outputs and sort what goes wrong into categories. Build evals for the failures you actually see, not generic scores.
- Do not over-tune. If you keep tweaking until the system passes every golden set case, you have fit the test, not real quality.
The golden set is a proxy for real quality. Passing all of it is not the goal; working for real users is.
Short line#
Golden sets cover the cases you thought of; production shows the rest. Watch implicit signals like rephrasing and escalation, error rates and feedback, turn each failure into a golden set case, and rerun evals. Start by reading real outputs, and do not over-tune to the golden set.
Key takeaways#
- Production finds the cases your golden set missed.
- Explicit feedback is direct but rare and biased toward strong reactions.
- Implicit signals (copy, rephrase, escalate) are high volume and hard to fake.
- Track error rates: no answer, timeouts, refusals that should not happen.
- Every production failure becomes a golden set case.
- Start eval work by reading 50 to 100 real outputs, and do not overfit to the golden set.