AI hallucinations: why they happen and how to reduce them

You ask an AI assistant for a study reference and get a title, authors and a year. All plausible. When you look for it, you find out the study does not exist. This is the best-known example of what is called an AI hallucination.

This article explains what the phenomenon is, why it happens according to the available research and what helps reduce it, for those who use AI and for those who build systems with it.

What a hallucination is

The generative AI profile from NIST, the US standards institute, prefers the term confabulation. It defines it as the production of confidently stated but erroneous or false content, popularly known as “hallucination” or “fabrication” (NIST AI 600-1, July 2024). The definition also includes responses that diverge from the request or the material provided, and responses that contradict what the system itself said earlier in the same conversation.

NIST also notes a caveat: some commentators consider that “hallucination” attributes human characteristics to a machine. In this article we use the popular term because it is the most searched, but it is worth remembering that the model does not “see” anything. It generates text.

A broad academic review of the topic, published by Huang and co-authors and revised in 2024, divides the hallucinations of large language models (LLMs) into two groups (Huang et al., 2023):

• Factuality hallucination. The content contradicts verifiable facts about the world, or invents facts that cannot be confirmed. Example: attributing a law to the wrong year, or citing a nonexistent study.
• Faithfulness hallucination. The content does not follow the instruction, contradicts the context provided or has inconsistent internal logic. Example: a summary that includes a clause that is not in the contract you sent.

The distinction is practical. The first type requires checking the world. The second requires checking the source document or instruction.

Why models hallucinate

To understand the causes, it helps to remember how an LLM works: it generates text by predicting, step by step, the next unit of text, called a token (see tokens in AI models and what LLMs are). NIST sums up the consequence: confabulations are a natural result of the way these models are designed, since they generate outputs that approximate the statistical distribution of their training data. This process can get things right, but it can also produce factually wrong or inconsistent content.

The review by Huang and co-authors organizes the causes into three areas.

Data. Training material contains errors and myths, which the model can reproduce. There are also knowledge limits: rare facts (the so-called long tail), outdated information and content that was never in the data.

Training. In the fine-tuning stage with examples, the model is pushed to always complete the answer, which makes it harder to decline or express doubt. In the human preference tuning stage, sycophancy can emerge: the tendency to please the evaluator, sometimes at the expense of the truth.

Generation. The randomness used to choose words is associated with a higher risk of hallucination, according to the review. The review also points to overconfidence and insufficient attention to the context provided.

The incentive to guess

A September 2025 paper by Adam Tauman Kalai and co-authors adds an explanation that goes beyond the technical (Kalai et al., 2025). The argument has two parts.

First, in pretraining, some errors are statistically expected. Arbitrary facts that appear only once in the data, such as the birthday of a little-known person, have no pattern the model can learn. The authors illustrate this with Kalai himself: asked three times about his birthday, a model gave three different dates, all wrong.

Second, and more importantly, the problem persists because most performance tests give points only for the right answer and zero for “I don’t know”. In the authors’ words, models are optimized to be “good test-takers”, and guessing under uncertainty improves the score. Their proposal is to change the scoring of the most widely used tests so as not to penalize admitting uncertainty. They also argue that hallucination is not inevitable in every system: a system that answers only what it knows and abstains on the rest does not hallucinate, even if it answers less.

How often this happens in practice

There is no single hallucination rate. It varies with the model, the subject, the language and the way the question is phrased. Studies give orders of magnitude for specific scenarios, always dated.

• Legal questions. A study published in the Journal of Legal Analysis in 2024 found hallucinations between 58% of the time, with ChatGPT 4, and 88%, with Llama 2, on specific, verifiable questions about US federal court decisions (Dahl et al., 2024). The authors also observed that the models often failed to correct wrong legal premises embedded in the question.
• Popular false beliefs. On TruthfulQA, from 2021, the best model tested was truthful on 58% of 817 questions, compared with 94% for humans, and the largest models were generally the least truthful (Lin et al., 2021).

These numbers describe the models and conditions of those studies. They cannot be used to estimate the performance of current tools. What they clearly show is that the problem is real in specialized fields and that a confident tone in the answer does not protect the user.

The concrete risk, according to NIST, arises when someone believes the false content and acts on it. The document cites as an example a confabulated summary of patient information that could lead to a wrong diagnosis.

How to reduce it: what works for users

No technique eliminates hallucinations. Anthropic’s documentation on the subject says so directly: the techniques reduce the problem but do not eliminate it, and critical information should always be validated (Anthropic, “Reduce hallucinations”). That said, some practices help.

1. Give permission to say “I don’t know”. Anthropic recommends explicitly allowing the model to admit uncertainty. This fits with the argument of Kalai and co-authors: if “I don’t know” is acceptable, guessing loses its advantage.
2. Provide the material and restrict use to it. When the answer should be based on a document, send the document and ask the model to use only that information.
3. Ask for direct quotes first. For long documents, Anthropic suggests asking for exact excerpts before the analysis and basing the answer on those excerpts.
4. Ask for each claim to be supported. After the answer, ask the model to find a supporting excerpt for each claim and remove anything without support.
5. Ask again and compare. If answers to the same question diverge, treat the divergent points as suspect.
6. Check what matters outside the AI. Names, numbers, dates and references should be checked against the original source.

How you phrase the request makes a difference. Clear instructions and enough context reduce the room for the model to fill gaps on its own. See our guide to prompt engineering.

How to reduce it: what works for system builders

For teams putting LLMs into products, there are more structured techniques.

Retrieve before generating (RAG). Retrieval-augmented generation combines the model with a search mechanism over a document base. The paper that coined the term, from 2020, showed that this combination produced “more specific, diverse and factual” text than an equivalent model without retrieval, on the tasks tested (Lewis et al., 2020). We cover the topic in detail in RAG and AI with documents.

RAG does not solve everything. The review by Huang and co-authors lists retrieval failures (ambiguous query, wrong document retrieved, poor splitting of texts) and generation failures: irrelevant excerpts can induce errors, and the model may trust what it “remembers” from training more than the retrieved document. Another study showed that models make better use of information at the beginning or end of a long context than in the middle (Liu et al., 2023). Organizing what goes into the context matters; see context and memory in AI.

Verify in stages. The Chain-of-Verification method has the model draft an answer, plan verification questions, answer those questions independently and only then produce the final version (Dhuliawala et al., 2023).

Measure consistency. SelfCheckGPT detects possibly invented passages by generating several answers to the same question and comparing them. The premise is that when the model knows the fact, the answers converge; when it invents, they diverge (Manakul et al., 2023). It has the advantage of not requiring an external knowledge base, and the disadvantage of costing several calls to the model.

Measure before you trust. Any of these techniques needs to be tested on your use case, with real examples. We show how to set up that evaluation in how to evaluate the quality of AI responses. In systems that act on their own, such as AI agents, an invented error can become an action, which calls for even more care.

Takeaways

• A hallucination is false or unsupported content presented with confidence. It can contradict facts about the world or the material you provided.
• It arises from the way models generate text, from gaps and errors in the data, from training stages that favor always answering and from tests that reward guessing.
• Rates vary widely by model and subject. Dated studies show the problem is serious in specialized fields such as law.
• To reduce it: accept “I don’t know”, provide sources, ask for verifiable citations, compare answers and check what matters outside the AI.
• No technique eliminates the problem. In health, legal, financial or security decisions, the final word belongs to a reliable source or a professional.