What this is

1. The short version

This is a short course about evaluating AI outputs.

When you ship an AI feature, somebody on your team eventually asks the same question. "But is it actually working?" Most teams don't have a clean answer. They have a few screenshots, a vibe, and a launch date. This course is about how to give the clean answer.

It's free, and it's read-only. No exercises. No drills. No capstone. You'll close the tab in an hour or two knowing what an eval is, the vocabulary that comes with it, and the smallest useful eval you can build for a real feature.

2. Who it's for

You, if you're an engineer who has shipped an AI feature, or is about to, and doesn't know how to measure whether it's working. That includes:

• Backend engineers who added an LLM call to a service and have no test for it.
• Product engineers shipping AI features under a deadline who keep getting asked "is this good?" and don't have a number to point at.
• Engineers brand-new to AI work who want to learn this before they ship the thing, not after.
• Tech leads who want vocabulary to push back when somebody says "the model just feels off this week."

You don't need to have used an eval framework. You don't need a stats background. You don't need to know what F1 score means. We'll get there.

3. What it isn't

It isn't a tutorial. You won't install anything. You won't write code in this course. You won't pick a framework.

It isn't a survey of vendor products. No tool comparisons. No screenshots of dashboards. Tools come and go. The thinking doesn't.

It isn't a deep dive on statistics. You'll see the trio (accuracy, recall, precision) defined plainly, with an example. If you want the math, the math is elsewhere.

It's a briefing. Enough to know what an eval is, enough to talk about evals with somebody on your team without bluffing, and enough to write your first one when you go back to your code.

4. How to read it

Front to back. The chapters build.

Chapter 1 defines what an eval is. Chapter 2 gives you the vocabulary. The later chapters cover designing eval sets, the main types of evals, how to read the numbers, and how evals work in production.

If you only read two chapters, read chapters 1 and 2. They're the ones that change how you think.

5. The next page

The next page is five honest questions. If most of them are yes, keep reading. If not, the page will tell you where to go.

That's it. Let's start.