AI Tools Field GuideChecking output the smart way
Check a draft by depth of risk, not line by line. Ask the model to hand you the claims and the references, check those, and spend the rest of your time on the things that would hurt if they were wrong.
Why checking is needed at all
A language model is not a knowledge store. It was trained on a vast amount of text and learned to continue text; it does not look things up, and it does not know what happened at your organization or what your instruction says unless you attached it. When it lacks something it fills the gap fluently, and the invented paragraph reads like the real ones. Web search does not fix this: a tool's search uses an index that is refreshed periodically, not the live internet, so it does not guarantee current information either. Two habits remove most of that risk before any checking: attach the source, and tell it that "I don't know" or "not in the source" is an acceptable answer.
Match the depth to the risk
| Output | How much checking | What that looks like |
|---|---|---|
| A rewording, an email to your own division | One read | Does it say what you meant? |
| A summary you will forward | Spot-check | Open three of the references; if all three hold, send it |
| A memo the CO signs | The claims list | Every claim traced to a line in the source before it goes up |
| Numbers in a brief | Recompute | A spreadsheet or the official table, not the model's arithmetic |
| Anything with legal, medical, pay or safety consequences | Full check | A person who did not draft it reads it against the source |
Make the model do the finding
- Ask for references"Put the paragraph number beside each item." Checking becomes opening a page and glancing, not hunting.
- Ask for the claims list"List every factual claim in this draft and where it comes from." Ten lines to check instead of three pages.
- Use a fresh chat as reviewerNew conversation, attach the draft and the source: "List every claim the source does not support." It has none of the first chat's commitments.
- Ask what it assumed"List the assumptions you made." Turns guesses into a checklist, and the guesses are where the errors live.
Design the check in
Nobody is going to check everything by hand, and a rule that depends on it will be quietly abandoned by the second week. So build the check into how you ask, and let the shape of the answer expose the error.
- Make the math a tool call"Use Python for every calculation, show the code and its output." A wrong number becomes a wrong program you can re-run, instead of a fluent sentence.
- Ground it in your own files"Answer only from the attached documents. Quote the passage that supports each claim. If there is no passage, write [UNSUPPORTED] instead of filling the gap."
- Ask for the citation as a quote and a locationNot a reference number: the sentence, plus the document and paragraph it came from. Then open one of them yourself.
- Chain of verification"List the claims. Now write the questions that would check each one." Answer those questions in a fresh chat that has never seen the draft, then paste the answers back and ask for the corrections.
- An adversarial pass"Argue the opposite and find every error", or hand the draft to a second vendor's model. Two labs' models rarely share a blind spot.
- Structured outputForce a table with a column for the source and the quote. A missing field is visible; a missing sentence in a paragraph is not.
A caution on the adversarial pass: a model arguing with itself is weaker than it looks, and the published comparisons find that asking the same question a few times and comparing the answers does better for less effort. An adversary only earns its keep when it has something the first pass did not: a source document, a different model, or a person who knows the ground truth.
Turning the temperature down is not on that list. It makes answers more repeatable and no more true, most chat windows do not expose it, and the studies find no reliable effect on invention either way.
The cross-model check, in practice
- Draft in one tool with the sources attached
- Paste the draft and the sources into a different vendor's careful model
- Ask it to mark every claim the sources do not support
- Fix what it found and rewrite
- A person who owns the product reads it last
The second model is not smarter. It was trained differently, so it fails differently.
What a program should check
Dates, sums, counts, IDs, and anything with a fixed format belong to a spreadsheet or a lookup, not to the model. Ask it to write the formula or the script, or to compute with a tool where the tool exists (Grok's skills), then read the result. A total that "looks right" and was never computed is the classic embarrassment; the next lesson, Every limit has a workaround, shows the routine for it.
The tells that something was invented
| The tell | What to do |
|---|---|
| A reference number, date or name you did not supply | "Where in the attachment is that?" If it cannot point to it, it invented it. |
| A citation you cannot open | Open it. A citation points at the proof; it is not the proof. |
| A number with no working shown | Ask for the inputs; recompute. |
| An answer that is suspiciously complete | "What is missing from the source?" |
Before you brief it
A slide is a draft with the references stripped off. Keep the referenced version in your notes, and for every number on the slide be able to say where it came from. If a claim came from the model and not from a source you opened, it comes off the slide.