The honest test

1. Five questions

Answer these honestly. The course is built for people who say yes to most of them.

1. Have you shipped, or are you about to ship, a feature that calls a language model?
2. Has somebody (your manager, your PM, your CEO, you) asked whether it's working, and you didn't have a clear answer?
3. Have you ever shipped a prompt change because the new one "felt better" on three examples you tried in a notebook?
4. Do you suspect your AI feature is worse for some users than others, and you don't have a way to check?
5. Are you tired of debating whether the model is good enough, and would rather show people a table?

If you said yes to three or more, you're in the right place. The course assumes that's roughly your situation.

2. If you said yes to none

You might be earlier than this course expects.

If you've never called a language model from code, start with [[zero-to-ai-engineer]] to get the lay of the land, then [[prompt-to-production]] to actually ship a small feature. Come back here when you have a thing in production and somebody starts asking whether it works.

If you're already past evals — running offline test suites, tracking quality in CI, comparing model versions weekly — this course will feel slow. The thinking here is the floor, not the ceiling.

3. What the course will not do

It will not teach you a specific framework. The patterns transfer; the framework names don't.

It will not tell you what the right accuracy number is for your feature. Nobody can tell you that without knowing your product, your users, and your tolerance for being wrong.

It will not promise that evals fix every quality problem. Some problems are upstream (your prompt is bad, your retrieval is bad, your training data is bad). Evals tell you which problem you have. Fixing it is still your job.

4. What you walk away with

By the end of these 12 units you'll have:

• A working definition of an eval that isn't borrowed from a vendor.
• The vocabulary the rest of the industry uses (accuracy, recall, precision, coverage, regression, ground truth, eval set).
• A concrete picture of the smallest useful eval, on a real feature.
• A clear sense of what to build first when you go back to your code.

That's the whole promise. Read on.