No prevention method removes hallucinations completely, so some will reach your checks. Catching them means testing the output against something outside the model: a source, a lookup, a test run or a person. This note covers five checks, what each costs and when to use it. Prevention itself is in reducing hallucinations.
Five checks#
Each check works on a different kind of output, so most systems combine two or three.
| Check | How | Best for |
|---|---|---|
| Claims against sources | Trace each claim to a retrieved source, by hand or with an LLM judge | RAG answers |
| Verify citations | Look up each cited source and confirm it says what the model claims | Answers that cite papers, cases or links |
| Run and test code | Execute the code against known inputs and read every line | Generated code |
| Sample several times | Ask the same question repeatedly and compare the answers | Factual questions with no source to check |
| Human review | A person approves the output before the end user sees it | High-stakes outputs |
Check claims against sources#
In a RAG (retrieval-augmented generation) system, you retrieve relevant documents and put them in the prompt. Every claim in the answer should then trace back to one of those chunks. A claim that appears in no chunk is a hallucination. You can spot-check this by hand, or automate it with an LLM judge: a second model call that checks each claim against the sources.
Verify citations#
If the model gives a citation, check that the source exists and that it says what the model claims. A fake citation looks real until you look it up, so the lookup is the whole check.
Run and test generated code#
Never trust generated code you have not run. Code that looks correct and has valid syntax can still be wrong. Run it, test it against known inputs and read every line before you ship it.
Sample several times#
Ask the same question a few times and compare the answers. When the answers differ on a factual question, the model is probably guessing rather than recalling something solid. The answers can differ at all because of temperature, which makes the model sample from its likely next tokens instead of always taking the top one.
Use human review for high stakes#
When a mistake is costly, a person checks the output before it reaches the end user. Automated checks cut the volume of outputs that need review, but they do not replace judgment on the ones that matter most.
The trade-off#
Every technique that reduces or catches hallucinations adds cost or complexity somewhere. The table lists where it lands.
| Technique | What it costs |
|---|---|
| RAG | You build and maintain a retrieval pipeline |
| Requiring citations | Longer outputs |
| Tools | Integrations and error handling |
| LLM judge or repeated sampling | Extra model calls, so more cost |
| Human review | Expensive at scale |
The engineering question is always how much a hallucination costs in your context. The rate you can accept decides how much mitigation is worth building.
A common trap#
Saying "we told the model not to hallucinate" is not a fix. Writing "don't make things up" in the system prompt has almost no effect. The model does not hallucinate on purpose. Plausible next-token prediction produces confident text whenever knowledge is missing.
The fix is architectural, not instructional: ground the answer, give the model tools, and check the output.
Short line#
Hallucination happens because the model aims for plausible text, not true text, and has no step that verifies itself. Ground answers, allow abstention, require citations and give it tools for facts. Then catch what slips through by checking claims against sources and testing generated code.
Key takeaways#
- Check claims against the retrieved sources, by hand or with an LLM judge. A claim in no source is a hallucination.
- Look up every citation: confirm it exists and says what the model claims.
- Run and test generated code. Valid syntax proves nothing.
- Disagreement across repeated samples signals guessing.
- Use human review where mistakes are costly.
- Every mitigation costs something. Match the effort to the cost of a wrong answer.
- Telling the model not to hallucinate does almost nothing. Fix the design instead.