Write the eval before you write the prompt

There is a specific way LLM projects stall. The demo works. Somebody changes a prompt to fix a reported failure. Something else breaks, quietly, and nobody notices for two weeks. After a few rounds of this, the team stops changing anything — and a feature nobody dares to modify is a feature that is finished, whether or not it is any good.

The missing piece is not a better model, a bigger context window or a fashionable framework. It is the ability to answer one question in under a minute: did that change make the system better or worse?

An eval suite is a test suite that tolerates ambiguity

Conventional tests assert exact equality. Language model outputs vary, so the assertion has to move up a level: not 'the output is this string' but 'the output cites a source that supports the claim', or 'the output refuses when the retrieved context is empty'.

That is harder than a unit test and considerably easier than people assume. Most of the value comes from thirty to fifty carefully chosen examples, not from thousands.

• Cases that currently work, so you notice when they stop
• Cases that currently fail, so you can tell when a change fixes them
• Adversarial cases — prompt injection, out-of-scope questions, empty retrieval
• Boring cases, which is where most real traffic actually lives

Grade the parts, not just the whole

A wrong answer has at least three possible causes: retrieval returned the wrong context, the model reasoned badly over correct context, or the output format broke downstream parsing. A single end-to-end score cannot distinguish them, so it cannot tell you what to fix.

Score retrieval separately — did the correct document appear in the top k? — and you will usually find retrieval is the problem. That is good news, because retrieval is far cheaper to improve than reasoning.

Put it in CI on day one

An eval suite run manually before releases becomes an eval suite run before some releases, and then before none. In CI, with a threshold that fails the build, it stays honest.

# Fail the build when quality regresses
- name: Run evals
  run: pnpm eval --suite golden --min-score 0.82

The threshold matters less than its existence. Set it just below current performance and raise it as the system improves.

What this buys you

Model portability, mostly. When a new model arrives, you run the suite and get a number within the hour instead of a fortnight of vibes-based comparison. Teams with eval suites upgrade models routinely. Teams without them are still running whatever they launched on, because nobody can prove a change would be safe.

You cannot improve what you cannot measure, and with language models you cannot even tell whether you have broken it.