Why 'looks right' isn't measurement

1. The trap

You change the prompt. You run it on three tickets. The outputs look fine. You ship.

Two days later somebody reports that urgent tickets are being marked as medium. You roll back. You read the diff. You blame the model. You promise yourself you'll be more careful next time.

The next time looks exactly the same.

This pattern shows up on every team that doesn't have evals. It's not a discipline problem. It's a measurement problem. Eyeballing doesn't scale, and it doesn't catch the things you weren't looking for.

2. Why three examples isn't enough

When you check three outputs by eye, you're sampling. The sample is tiny and it's biased.

The sample is tiny because three is three. If your feature gets one input wrong in twenty, you have an 86% chance of seeing all three look fine — even though one in twenty is going to embarrass you in production.

The sample is biased because you picked the three. You probably picked easy ones, or representative ones, or the same three you always pick because they're already in your scratch buffer. You did not pick the gnarly ones, because if you had a gnarly ones folder you wouldn't be doing this by hand in the first place.

3. What you don't see when you eyeball

This is the bigger half of the problem. Eyeballing only catches errors you're already looking for.

If you're checking category, you'll notice category errors. You'll glance at priority and it'll look about right. You will not notice that priority is now systematically one notch lower than it was last week, because nobody is computing the average.

If you're checking the summary, you'll notice it sounds fluent. You will not notice it's now silently inventing a customer name in 5% of cases, because you've been reading the summary and not cross-referencing the input.

The errors that eat features in production are almost always errors nobody was watching for. Evals catch them because they grade every case on every dimension every time. Eyeballing catches them only by accident.

4. The cost of being wrong about being right

There's a worse failure mode than catching the wrong errors. It's concluding the feature is working when it isn't.

You shipped a prompt change. You eyeballed three outputs. They looked good. You broadcast in Slack that the new prompt is better. Your team trusts you. They build on top of it. Marketing tells customers the AI summaries are now more accurate.

A week later, the support inbox starts filling up. The new prompt was worse on a population you didn't sample. The cost isn't just the bug. It's the lost trust — both internally ("they said it was better") and with customers ("you told us this was fixed").

This is why eval discipline matters more for AI features than for traditional code. Traditional code either compiles or it doesn't. The wrongness is in the open. AI features look fluent even when they're wrong, and fluent wrongness gets shipped, broadcast, and built on top of, until somebody actually measures.

5. What a table buys you that eyeballing doesn't

Even the five-by-three table from the last unit gives you four things eyeballing can't.

Repeatability. You can run the same five cases on tomorrow's prompt and last week's prompt and compare. Eyeballing doesn't compare — it just reassures.

Coverage. Five cases chosen to cover billing, technical, account, general, and an urgent edge case touch more ground than three cases pulled out of your scratch buffer.

Multi-dimensional grading. The table grades category, priority, and email separately. You see "priority is the one regressing" instead of an overall feeling that things are fine.

A trail. Tomorrow's you can read today's table. Eyeballing leaves no trail. You'll have to redo the work and you'll redo it differently.

6. The honest replacement

The replacement for eyeballing isn't "stop looking at outputs." Looking at outputs is still part of the job — that's how you discover new failure modes, write new cases, and build intuition.

The replacement is: when you look at an output and form an opinion, write a case that captures the opinion. If you saw that summary and thought "it shouldn't be inventing a customer name," add a case where the right answer has no customer name and the wrong answer does. Now your intuition is in the eval set. The next time the model regresses on it, you'll find out from the table, not from a customer complaint.

That's the move. Looking is fine. Looking without writing it down is the trap.

7. What's next

The last unit of this chapter names the mindset shift. Engineers who write evals stop arguing about whether an AI feature feels better. They start showing each other tables. That's the whole chapter, compressed.