Eval set vs test set vs production traffic
1. Three populations that get confused
When somebody says "we tested it on 200 cases," they could mean three different things. The distinction matters because each one tells you something different about your feature.
The three populations are:
- Eval set — the cases you iterate against while developing.
- Test set — a held-out group of cases you don't look at until the end.
- Production traffic — the actual inputs hitting your feature in the real world.
Same shape (input plus expected output, mostly). Different jobs. Different rules about when you're allowed to look at them.
2. The eval set
Your eval set is the working set. You write it. You read it constantly. You add cases when new failure modes show up. You iterate against it the same way you'd iterate against a unit test suite.
Every prompt change, every model swap, every refactor — you rerun the eval set first. The numbers on the eval set are your feedback signal during development.
The eval set has one big problem. Because you look at it constantly and tune your changes against it, you'll inevitably overfit to it. You'll discover that a specific phrasing makes case 12 pass, and you'll bake that phrasing into the prompt even if it makes other (un-tested) cases worse. This is the same overfitting problem ML teams have worried about for years. It's real for prompt engineering too.
You can't avoid overfitting to the eval set. You can only catch it, which is what the test set is for.
3. The test set
The test set is cases you've intentionally not looked at.
You write them, label them, lock them in a drawer. You don't iterate against them. You don't tune your prompt to make them pass. You rerun them rarely — once a sprint, once before a release, once when something feels off.
When you do rerun them, the number on the test set tells you whether you've overfit to the eval set. If your eval-set score is 0.92 and your test-set score is 0.71, you've been tuning against a specific set of cases and the feature is worse than you think on cases you haven't seen.
The test set is also called a held-out set or a holdout. The discipline is the same: don't peek.
4. Production traffic
Production traffic is the inputs your feature is actually receiving in the wild.
These are not labeled. They don't have an "expected output" column, because nobody wrote one. You can't compute accuracy against them directly. What you can do is sample them — pull 50 random inputs from yesterday's traffic, hand-label them, and use that as a check on whether your eval set still looks like the real world.
Production traffic is the source of truth about what your feature is actually doing. The eval set is a proxy. The test set is a proxy. Both proxies decay over time as the world changes. Sampling production keeps them honest.
5. The comparison
| Who labels it | When you look at it | What it tells you | |
|---|---|---|---|
| Eval set | You and your team, ahead of time | Constantly during development | How your changes affect quality on known cases |
| Test set | You and your team, ahead of time | Rarely; before releases or to spot-check | Whether you've overfit to the eval set |
| Production traffic | Nobody, by default; you can sample and label | Continuously, in the background | What the world is actually asking, and whether your evals still match it |
These three are populations you choose how to work with, not properties of your data. The same case can move between them over time. A case from production that you labeled and added to the eval set is now part of the eval set, not production.
6. The contamination problem
This is the failure mode you most want to avoid.
Contamination is when a case you intended to be in the test set leaks into the eval set, or worse, into the prompt itself.
The most common version: you add a case to the test set, then a week later forget you added it and use the same case as an example when you're iterating on the prompt. Now the prompt was tuned to handle that exact case, and the test-set score on it doesn't mean what you think it does.
A subtler version: your few-shot examples in the prompt got copied from a real production ticket that's also sitting in your eval set. The model isn't being evaluated on those cases. It's being shown the answer.
The fix is mechanical. Tag every case with which set it belongs to. Tag the few-shot examples in your prompt. Run a check that no case ID appears in two places. It's boring infrastructure, and skipping it is how teams end up with eval numbers that don't match reality.
7. What proportions
A rough starting allocation for a feature like the ticket extractor:
- Eval set: 30–80 cases. Big enough to be useful, small enough to read.
- Test set: 20–50 cases, held-out, untouched.
- Production sample: 20 cases per month, freshly drawn and labeled.
If you can't afford that much labeling, drop the test set first and keep the production sampling. The production sample is what protects you against the eval set drifting out of sync with reality. The test set is a check you can re-add later.
8. What's next
That closes chapter 2. You now have the vocabulary. Accuracy, recall, precision. Coverage. Regression, degradation, drift. Ground truth and the labeling problem. The three populations.
The rest of the course builds on this. Chapter 3 is about designing the eval set itself — what cases to include, how to write them, how to keep them fresh. Chapter 4 is about the main types of evals (exact match, semantic match, rubric, LLM-as-judge) and when each one fits. The vocabulary you just read is what those chapters will use.