AI Tools Field GuideEvidence and evaluation
A benchmark score is not evidence that a model works for your task. Thirty of your own inputs, with the answers fixed in advance, is.
- Write the task down
- Collect thirty real inputs
- Fix the reference answers
- Run every candidate
- Score blind
- Decide, once
An evaluation is this strip, in this order. The order matters: the references are fixed before any candidate is run.
Benchmarks
A benchmark is a public test: a fixed set of questions with known answers, scored the same way for every model. Vendors report their scores at launch and the scores are what the headlines compare. Three cautions come with them.
- It is the vendor's number until somebody outside reproduces it, and reproductions often come in lower.
- The questions may be in the training data. Public tests leak into the corpus; a model can have seen the answers. This is called contamination and it inflates scores without improving the model.
- It is not your task. A graduate-level science score tells you little about whether the model drafts a read-ahead your commander would accept.
Use benchmarks to shortlist: a model that scores badly on a test close to your work is probably not worth your time. Never use them to choose. The spec sheet page has the rest of the reading rules.
A held-out test of your own
The evidence that counts is the model's performance on your task, measured on inputs it has not seen, against answers you fixed in advance. The discipline is the same one a training run uses to pick a checkpoint, and it fits on an index card:
| Decide before you run anything | Example for a read-ahead drafter |
|---|---|
| The task, in one sentence | "Draft a one-page read-ahead from the meeting's three source documents" |
| The inputs: real, recent, including the awkward ones | Thirty past meetings with their documents |
| The reference: what you would have accepted | The read-ahead the action officer sent, tidied |
| The metric | A four-point scale: send as is, minor edits, major edits, unusable |
| The baseline | Whatever you do today: the current model, or a person's first draft |
| The acceptance rule | "At least as many send-as-is as the baseline, and no unusable" |
Thirty is not many, and what you read from thirty is coarse: a difference of two or three either way is noise. That is fine. The test catches the candidate that is clearly worse and stops the conversation being about impressions. It will not rank candidates to the decimal. Keep the set, and run it again whenever the vendor updates the model or somebody edits the instructions. A set you keep and rerun is worth more to your shop than any published score, and it costs an afternoon.
Human judging, done blind
Numbers do not capture whether a draft reads well, and for most staff work reading well is the job. So the score comes from a reader, and the reader has to be protected from knowing which model wrote what, because everyone, including you, prefers the one they expected to prefer.
The form that works is blind pairwise comparison: the same input to two candidates, the two outputs side by side, the order randomized so the left one is not always the same model, and the judge picks the better one or calls a tie. The labels are revealed only in the tally. We built this comparison for our own weather image models, because the metrics said two versions were equal and our eyes said otherwise. The blind tally settled it, more than once against the version we had expected to win. Two judges are better than one, and their agreement rate is itself a finding: if two competent people disagree on a third of pairs, the difference between the models is smaller than the difference between readers.
The papers behind this guide
Every "why" in the earlier tiers traces to one of these. Several 2025 and 2026 items are preprints or vendor reports rather than peer-reviewed work, and the guide says so where it leans on them.
| Claim in this guide | Source |
|---|---|
| The transformer, the architecture under every LLM | Vaswani et al., "Attention Is All You Need," NeurIPS 2017 |
| Examples in the prompt work (in-context learning) | Brown et al., "Language Models are Few-Shot Learners," NeurIPS 2020 |
| Stepwise reasoning improves accuracy (now built into reasoning models) | Wei et al., "Chain-of-Thought Prompting," NeurIPS 2022 |
| RAG | Lewis et al., "Retrieval-Augmented Generation for Knowledge-Intensive NLP," NeurIPS 2020 |
| Material in the middle of a long context is used less reliably | Liu et al., "Lost in the Middle," TACL 2024 |
| Effective context is often far shorter than advertised | Modarressi et al., "NoLiMa: Long-Context Evaluation Beyond Literal Matching," 2025 (preprint) |
| Hallucination is structural; RAG reduces but does not eliminate it | Ji et al., "Survey of Hallucination in Natural Language Generation," ACM Computing Surveys 2023, and 2024–2025 RAG evaluations showing residual errors when retrieval returns wrong or partial chunks |
| Uncertainty can be measured by sampling and checking agreement | Farquhar et al., "Detecting hallucinations in large language models using semantic entropy," Nature 2024 |
| Models guess because guessing was rewarded | Kalai et al., "Why Language Models Hallucinate," OpenAI, 2025 (vendor report) |
| Prompt injection is a real attack class | Greshake et al., "Not what you've signed up for," AISec 2023 |
Where this guide is least sure
- Which project and agent features are enabled per vendor comes from press and vendor statements as of 16 September 2026. Confirm against the product you use.
- "1 token ≈ ¾ word" is an English-prose average. It is worse for code, tables, and other languages.
- Compaction rules are vendor-specific and unpublished. This guide teaches the mechanism, not the thresholds.
What to do with this
Build the thirty-input set for the one task your shop uses these tools for most. It takes an afternoon and it will outlive every model you test on it. Next: the discipline for the most demanding use, Coding with models.