AI models hallucinate because they are trained to produce plausible text and are scored in ways that reward a confident guess over admitting uncertainty. You can reduce it a lot by giving the model the source material, explicitly allowing "I don't know", requiring quotes for each claim, and verifying anything that matters.
"Hallucination" is the usual name for output that sounds right but is false or unsupported. Anthropic's guide defines it as text that is "factually incorrect or inconsistent with the given context". It shows up as invented facts, wrong numbers, non-existent citations, function names that do not exist in a library, or a confident summary of something the document never said.
Why language models make things up
A 2025 paper by researchers at OpenAI and Georgia Tech, Why Language Models Hallucinate (Kalai, Nachum, Vempala and Zhang), gives the clearest explanation from a model vendor. Its central claim is that "language models hallucinate because the training and evaluation procedures reward guessing over acknowledging uncertainty." The argument has two parts.
1. Some facts cannot be learned from the training data
Pretraining teaches a model to predict likely text. For facts that follow patterns (spelling, grammar, common knowledge repeated thousands of times) that works well. For arbitrary facts that appear rarely, such as a specific person's birthday, there is no pattern to learn. The paper uses the share of facts seen exactly once in training (the "singleton rate") to estimate a floor: if 20% of birthday facts appear exactly once in the pretraining data, a base model should be expected to hallucinate on at least 20% of birthday questions.
The paper's own example: the authors asked a model for one author's birthday, instructing it to answer only if it knew. Across three attempts, the model (DeepSeek-V3) gave three different wrong dates.
2. Tests reward guessing
Most benchmarks grade answers right or wrong. An "I don't know" scores zero, exactly like a wrong answer, so a model that always guesses scores higher on average than one that admits uncertainty. The paper compares this to a student facing a multiple-choice exam: guessing when unsure maximises the expected score. Models tuned to do well on such tests learn the same habit. The bluffs tend to be specific and confident (a precise date rather than "sometime in autumn"), which makes them hard to spot.
The authors' proposed fix is on the evaluation side: change how existing benchmarks score abstentions so models are no longer penalised for honest uncertainty. That is a change for model builders. As a user, you can change the incentives inside your own prompt.
Other situations that make hallucination more likely
- Questions about things the model has no data on: very recent events, private documents, niche topics, or details of your own codebase it has never seen.
- Long inputs: the Lost in the Middle study (Liu et al., 2023) found models were less accurate at using information placed in the middle of a long context than at the start or end, which leaves room for the model to fill gaps.
- Leading questions: "Which section of the contract allows early termination?" presumes such a section exists.
- Pressure to produce something: requests for a fixed number of items ("give me 10 studies") when fewer real ones exist.
How to reduce hallucinations
The techniques below come from Anthropic's "Reduce hallucinations" guide and its prompting best practices. They work with other models too, because they address the causes above rather than a quirk of one model.
1. Give the model the source material
Do not ask a model to recall what it can read. Paste the document, the error log, the API reference or the data, and ask about that. Anthropic's guide calls this external knowledge restriction: explicitly instruct the model to use only the provided documents and not its general knowledge.
Answer using only the information in <docs>. Do not use outside knowledge.
<docs>
{{DOCUMENTS}}
</docs>
Question: {{QUESTION}}2. Explicitly allow "I don't know"
This directly counters the guessing incentive. Anthropic's guide says giving the model permission to admit uncertainty "can drastically reduce false information". Its example prompt ends with: If you're unsure about any aspect or if the report lacks necessary information, say "I don't have enough information to confidently assess this."
Give the model an exact phrase to use. That makes "I don't know" answers easy to detect in code.
3. Ask for quotes first, then the answer
For long documents (Anthropic's guide gives over 20k tokens as the point where this matters), ask the model to extract word-for-word quotes relevant to the question before answering, and to base the answer only on those quotes. A quote is easy to check against the source; a paraphrase is not.
1. Extract the exact quotes from <policy> that are relevant to data retention.
If there are none, write "No relevant quotes found."
2. Using only those quotes, answer: how long are customer records kept?
Refer to the quotes by number.4. Make every claim cite its source, and retract what cannot be cited
Anthropic's guide suggests a self-check step: after drafting, the model finds a supporting quote for each claim, and removes any claim it cannot support. Its press-release example asks the model to mark each removal with empty [] brackets, so you can see what was dropped.
5. Ask the same question more than once
The guide's "best-of-N verification": run the same prompt several times and compare the answers. Facts the model actually knows tend to come back the same; invented details tend to change between runs, just like the three different birthdays in the OpenAI paper. A text diff makes the differences obvious; larely's diff checker compares two outputs side by side.
6. Do not ask leading questions
Ask "Does the contract allow early termination? If so, quote the clause" instead of "Which clause allows early termination?". Ask for "up to 10 studies, or fewer if you are not certain they exist" instead of exactly 10.
7. For code, make the model read before it answers
Anthropic's prompting guide includes a sample instruction for coding agents: never speculate about code you have not opened, and read the relevant files before answering. Asking a model to work from the actual source, rather than from memory of a library's API, prevents a common failure: calling functions that do not exist.
Verify what matters
None of these techniques bring the error rate to zero. Anthropic's guide ends with exactly that caveat: the techniques "don't eliminate them entirely", so "always validate critical information, especially for high-stakes decisions." A practical rule:
| Type of output | How to check it |
|---|---|
| Quotes from your documents | Search the source for the exact text |
| Citations and links | Open every one; confirm it exists and says what was claimed |
| Numbers and calculations | Recompute them, or ask for the working and check each step |
| Code | Run it and its tests; check that imported functions exist in the version you use |
| Facts about the world | Confirm with a primary source (official docs, the original paper, the standard) |
Structure your prompt so these checks are easy: quotes, citations and step-by-step working all give you something concrete to verify. How to write a good prompt covers the rest of the prompt structure.
Frequently asked questions
- Will a newer or bigger model stop hallucinating?
- It will usually hallucinate less, but not never. Anthropic's own guide says its techniques significantly reduce hallucinations without eliminating them, and tells you to validate critical information. The OpenAI-authored paper Why Language Models Hallucinate argues some errors are statistically unavoidable for facts that appear rarely in training data.
- Does setting temperature to 0 prevent hallucinations?
- No. A low temperature makes the model pick its most likely next token more consistently, which reduces randomness, but if the most likely answer is wrong it will be consistently wrong. Several current hosted models also no longer accept custom temperature values. Grounding the model in sources and allowing it to say it doesn't know are more effective.
- Why does the AI invent citations and URLs?
- A citation is just more text to the model. It has learned what references look like, so when asked for one it can produce a plausible-looking author, title and link without having any real document behind it. Ask it to quote from documents you supplied, and open every link it gives you before relying on it.
- Is hallucination the same as the model lying?
- No. Lying implies intent. A hallucination is an error: the model produces fluent text that is not supported by its training data or by the context you gave it. The practical result for you is the same, which is why important answers need checking.