AI hallucinations: what they are, why they happen and what to do

What is an AI hallucination?

An AI hallucination is output from a generative AI tool that is confidently presented but false, fabricated or not supported by the input it was given. NIST's Generative AI Profile uses the term confabulation for the same thing and describes it as generative systems that generate and confidently present erroneous or false content. NIST includes in that definition outputs that diverge from the prompt or other input, or contradict statements made earlier in the same conversation.

The word hallucination is a metaphor and a slightly misleading one. The tool is not seeing things. It is producing text that is statistically plausible for the context, and sometimes plausible text is wrong. That is why a hallucination rarely looks like an error. It looks like a normal, well-written answer.

Why does ChatGPT hallucinate?

ChatGPT and similar tools hallucinate because they are built to generate likely continuations of text, not to retrieve verified facts. NIST describes confabulation as a natural result of the way generative models are designed: they approximate the statistical distribution of their training data, and while that can produce accurate output, it can also produce output that is factually wrong or inconsistent. NIST notes the problem is particularly relevant to open-ended, long-form answers and to questions that need specialist expertise.

A 2025 research paper by Kalai, Nachum, Vempala and Zhang offers a second part of the explanation. The authors argue that training and evaluation procedures tend to reward guessing over admitting uncertainty, because a model that guesses scores better on many benchmarks than one that says it does not know. In their framing, models are optimised to be good test-takers. Whatever the precise mechanism, the practical lesson for an employee is the same: the tool's confidence tells you nothing about whether a statement is true.

Hallucinations are more likely when you ask about something obscure, recent, highly specific or outside the material you supplied: an exact figure, a named person's role, a quotation, a court case, a product's current price. They are less likely, though still possible, when the tool is asked to work only from a document you have pasted in.

What do AI hallucinations look like at work?

At work, hallucinations usually look like small, specific details that sound right. The list covers the forms employees most often meet. Learning to recognise the shapes makes them easier to catch.

What happens when a hallucination is not caught?

The best-known public example is Mata v. Avianca, decided in the US District Court for the Southern District of New York on 22 June 2023. Lawyers filed a court document that included non-existent judicial opinions with fake quotes and fake citations created by ChatGPT. The court found bad faith and imposed a sanction of 5,000 US dollars on the lawyers and their firm, jointly and severally, and required them to write to their client and to the judges whose names had been falsely attached to the invented opinions. In weighing the sanction, the court noted that the firm had arranged mandatory continuing legal education for its lawyers and staff on technological competence and AI tools: training, in other words, after the fact.

The lesson is not that lawyers are careless. It is that a fabricated citation looks exactly like a real one, and the person relying on it skipped the one step that would have caught it: opening the source. Most workplace hallucinations are smaller, such as a wrong delivery date in a customer email or an invented statistic in a pitch deck, but the failure is the same.

Are AI hallucinations still a problem?

Yes. Vendors have added features that help, such as web search with citations and the ability to answer from uploaded files, but none of them claims to have eliminated hallucinations. Anthropic's own documentation for Claude says that its recommended techniques reduce hallucinations but do not eliminate them entirely, and advises validating critical information, especially for high-stakes decisions. Google's help pages for Gemini in Gmail and Sheets warn that Gemini features may suggest inaccurate or inappropriate information.

Treat any statement that a tool no longer hallucinates with suspicion. A better question is which tasks a particular tool handles reliably enough, with which checks, for your purpose.

How can you reduce AI hallucinations?

You reduce hallucinations by narrowing what the tool has to invent: give it the source, tell it to use only that source, allow it to say it does not know, and ask it to show evidence for each claim. The techniques below come from vendor guidance, including Anthropic's documentation on reducing hallucinations, adapted for non-technical work. None of them removes the need to check.

Example prompt combining several techniques: Using only the policy text below, list the three rules that apply to staff travel booked within 48 hours of departure. For each rule, quote the exact sentence it comes from. If the policy does not cover something, write Not stated. Do not use general knowledge.

What does catching a hallucination look like in practice?

Consider a fictional 25-person dental supplies distributor, Northgate Supply. A sales coordinator pastes in the company's two-page returns policy and asks an assistant: What is our returns window for opened products, and is there a restocking fee? The supplied policy says unopened items can be returned within 30 days, opened items only if faulty, and says nothing about restocking fees.

The first answer: Opened products can be returned within 14 days, subject to a 15 percent restocking fee. Both details are invented. Nothing in the policy mentions 14 days or a fee. They are plausible because many returns policies contain them, which is exactly how a statistically likely answer goes wrong.

The coordinator reruns the question with the techniques above: use only the policy, quote the sentence, write Not stated if absent. The second answer: Opened items: returnable only if faulty (quote: Opened items may be returned only where the product is faulty). Restocking fee: Not stated. That answer can be checked in seconds against the document, and the gap about fees goes to the person who owns the policy rather than into a customer email.

What should an employee do after spotting a hallucination?

Remove or correct the false statement, then decide whether anyone has already relied on it. If the output was only a draft, fix it and move on, noting in your review that the tool invented that detail. If it has already been sent, shared or used in a decision, tell the person responsible so they can correct it. Do not quietly patch a document others have already read.

It also helps to tell colleagues which kinds of question produced the error. A team that shares examples of what went wrong learns faster than one where each person discovers the same failure separately. The step-by-step method for checking a draft is in the guide to fact checking AI output.

When is a bad AI answer not a hallucination?

Not every bad answer is a hallucination, and the fix differs. If the tool gives an outdated answer because the information changed after its training, the fix is a current source. If it gives a biased or one-sided answer, the issue is framing and fairness rather than invention. If it follows a vague prompt to a reasonable but unwanted result, the fix is a better brief. And if someone deliberately uses AI to mislead, that is a conduct matter, not a training one.

This guide also cannot tell you that a given tool is safe for a particular high-stakes task. For legal, medical, financial or employment decisions about individuals, use a qualified reviewer regardless of how the output was produced.

Sources and scope

NIST, AI 600-1: Generative AI Profile, July 2024 ↗

Kalai, Nachum, Vempala and Zhang, Why Language Models Hallucinate, arXiv, 4 September 2025 ↗

Anthropic, Claude documentation: Reduce hallucinations, accessed 16 October 2026 ↗

Mata v. Avianca, Inc., 678 F.Supp.3d 443 (S.D.N.Y., 22 June 2023), copy hosted by CLLA ↗

Gmail Help, Collaborate with Gemini in Gmail, accessed 16 October 2026 ↗

Google Docs Editors Help, Collaborate with Gemini in Google Sheets, accessed 16 October 2026 ↗