Evals Are the New Unit Tests

Traditional software fails deterministically. Given the same input, the same code path produces the same output every time. That determinism is what makes a unit test meaningful: run it once, trust it forever. AI features do not behave that way. Temperature, sampling, retrieval context, tool ordering, and model updates all move the answer around. A test that passes today can fail tomorrow without a single line of code changing. Evaluations are the tool built for that world.

Why unit tests are necessary but not sufficient

Unit tests still matter for the deterministic scaffolding around your AI: schema validation, tool wiring, retry logic, cost accounting. But the moment a model is in the loop, the interesting question is no longer "did this function return the right value" but "did this system, across a distribution of realistic inputs, produce outputs that satisfy the behavior spec." That is an evaluation, and it requires different infrastructure.

The spec-to-eval-to-deploy loop

Spec-driven development treats the specification as the source of truth. Evaluations are how that spec becomes executable. The loop looks like this:

1. Write the spec. Behavior, inputs, outputs, invariants, non-goals, failure modes.
2. Write the evals before the prompt. Golden examples for the happy path, adversarial cases for the failure modes, rubrics for the fuzzy criteria.
3. Choose an implementation. Prompt, model, tool chain, retrieval strategy. Now these are tunable parameters below the spec, not the spec itself.
4. Run the evals as a gate. Every change to prompts model swaps, or retriever tuning runs the harness. Regressions block deploy.
5. Watch production. Sample real traffic, grade it against the same rubrics, promote surprising failures into the eval set.

What belongs in a golden dataset

• Representative happy-path cases. The queries you designed the feature for, at the volume you expect.
• Every failure mode you have ever seen in production.Every incident becomes an eval case. Never regress a bug twice.
• Adversarial and boundary inputs. Empty strings, contradictory instructions, prompt injections, out-of-scope requests. Treat these like fuzz tests for text.
• Cases where the correct answer is "I do not know."The willingness to abstain is a first-class quality signal for AI.
• Long-tail edge cases from real users. Sampled and reviewed weekly. Production is the only oracle you have not yet fully lied to.

Rubrics: how to grade something fuzzy

Not every eval has a single correct answer. Summaries, drafts, and conversational replies need rubrics. A useful rubric is short, concrete, and independently scoreable: "preserves every named entity from the source," "does not exceed 120 words," "never prescribes a medication dosage," "cites at least one source when making a factual claim." Rubrics can be graded by a stronger model, by a human, or by a hybrid. What matters is that the score is reproducible and comparable across runs.

Regression gates that survive model swaps

The most valuable property of a good eval harness is that it lets you swap models without fear. A new frontier model drops on Tuesday, you point the harness at it Wednesday, and by Thursday you know whether it wins, loses, or draws on every criterion that matters to your product. Teams without evals cannot make that call. They swap on vibes, argue about it in review, and roll back after the first customer complaint.

Cost, latency, and safety as first-class eval dimensions

Quality is not the only axis. A model that answers correctly but five times slower and ten times more expensive is not a win. Neither is one that scores well on accuracy but leaks personal health information one time in a thousand. Serious eval harnesses report a full scorecard: accuracy, latency p50/p95, cost per request, refusal rate, safety violations. Deploy decisions look at all of it.

Where teams get it wrong

• Writing evals last. Same failure mode as writing the spec last. If the eval was not written before the implementation, the implementation is what defines the target.
• Grading by hand forever. Human review is essential for calibration and rubric design, but it does not scale. Automate what you can and reserve humans for the cases where automation disagrees with itself.
• Optimizing for the eval set. Evals that never change become the new leaderboard. Rotate cases, add new failure modes from production, and hold out a fresh sample the model has never seen.
• Ignoring drift. The same prompt on the same model can behave differently after a provider-side update. Rerun the harness on a schedule, not just on deploy.

Why this matters for mission-critical AI

In healthcare, public safety, and any regulated domain, "we tested it and it worked" is not an acceptable answer. Regulators, boards, and customers all want to see the evidence. An evaluation harness is that evidence. It turns a probabilistic system into something you can audit, compare, and defend. It is the discipline that lets AI graduate from demo to deployment.

The bottom line

If your AI feature does not have a harness, it does not have a quality bar. It has a hope. Build the evals first, write the spec that they enforce, and let the implementation be the cheapest part of the pipeline. That is what shipping AI you can trust actually looks like.