An LLM hallucination is a plausible but false statement that a language model presents as fact, such as a court case that does not exist or a refund policy the company never had. Two of the best documented examples are the six non-existent judicial opinions, all generated by ChatGPT that lawyers put before a federal court in Mata v. Avianca in 2023, and the Air Canada chatbot that told a grieving customer he could claim a bereavement fare after travel, for which a tribunal then held the airline liable. Models hallucinate because training and evaluation reward a confident guess over an honest "I don't know", and while you cannot switch that off, a system built around the model can catch much of the damage.
What is an LLM hallucination?
OpenAI defines hallucinations as plausible but false statements generated by language models, or instances where a model confidently generates an answer that isn't true. Researchers split them by what the output conflicts with. A widely cited survey distinguishes intrinsic hallucinations, output that contradicts the source content, from extrinsic hallucinations, output that can neither be supported nor contradicted by the source.
A 2024 paper in Nature narrowed the term further. It focused on confabulations, claims that are both wrong and arbitrary, meaning the answer is sensitive to irrelevant details such as random seed, and separated them from cases where a model is consistently wrong because it was trained on erroneous data. The distinction matters for fixing them, as the last sections show.
What are real examples of LLM hallucinations?
Six court cases that never existed (Mata v. Avianca, 2023)
In a personal injury suit against Avianca in the Southern District of New York, the plaintiff's lawyers filed an opposition to Avianca's motion to dismiss citing decisions such as "Varghese v. China Southern Airlines". Judge P. Kevin Castel's sanctions opinion found that the lawyers submitted non-existent judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatGPT. When the lawyer who ran the research asked the chatbot whether the cases were real, ChatGPT responded that it had supplied real authorities that could be found through Westlaw, LexisNexis and the Federal Reporter. He testified that he just never thought it could be made up.
On June 22, 2023 the court imposed a penalty of $5,000 jointly and severally and ordered letters to each judge falsely identified as the author of the fake "Varghese", "Shaboon", "Petersen", "Martinez", "Durden" and "Miller" opinions. The same day it granted Avianca's motion to dismiss. The judge was careful about where the fault lay: there is nothing inherently improper about using a reliable artificial intelligence tool for assistance, but existing rules impose a gatekeeping role on attorneys to ensure the accuracy of their filings.
An airline chatbot that invented a refund rule (Moffatt v. Air Canada, 2024)
A customer booking a flight after a death in the family asked Air Canada's website chatbot about bereavement fares. It told him he could submit his ticket for a reduced bereavement rate within 90 days of the date the ticket was issued by completing a refund application form. The airline's actual policy page said the bereavement policy does not apply to requests for bereavement consideration after travel has been completed, and it refused the refund.
Air Canada argued it could not be held liable for information provided by its chatbot, which the tribunal said amounted to suggesting the chatbot is a separate legal entity that is responsible for its own actions. British Columbia's Civil Resolution Tribunal rejected that in 2024 BCCRT 149: it should be obvious to Air Canada that it is responsible for all the information on its website. It ordered Air Canada to pay $650.88 in damages, $36.14 in pre-judgment interest and $125 in tribunal fees, 812.02 Canadian dollars in total. The sum was small. The principle was not: the company owns what its bot says.
Wrong facts about a named researcher (OpenAI, 2025)
OpenAI opened its own hallucination research with a test on one of its authors. Asked for the title of Adam Tauman Kalai's PhD dissertation, a widely used chatbot confidently produced three different answers, none of them correct. Asked for his birthday, it gave three different dates, likewise all wrong. Obscure personal facts are exactly the kind of detail a model cannot learn from patterns.
Code that imports packages that do not exist (2024 study)
A study of 16 code-generating models analyzed 576,000 code samples and found that the average percentage of hallucinated packages is at least 5.2% for commercial models and 21.7% for open-source models, including 205,474 unique examples of hallucinated package names. The authors treat this as a supply chain threat, a novel form of package confusion attack: if an attacker registers a name the models keep inventing, developers who trust the suggestion install the attacker's code.
Legal questions answered wrongly most of the time (2024 study)
Researchers asked models specific, verifiable questions about random federal court cases and found legal hallucinations occurring between 58% of the time with ChatGPT 4 and 88% with Llama 2. They also found that models often fail to correct a user's incorrect legal assumptions, and cannot always tell when they are hallucinating. The study was published in the Journal of Legal Analysis in 2024, and it tested models of that period, so treat the rates as a record of the problem rather than a score for today's models.
A medical answer that changes between runs (Nature, 2024)
The Nature paper's example: asked "What is the target of Sotorasib?", a model sometimes answers KRASG12 'C' (correct) and other times KRASG12 'D' (incorrect) despite identical instructions. Same question, same model, different answer, and nothing in either response signals doubt.
Why do LLMs hallucinate?
OpenAI's 2025 paper gives two reasons. The first is pretraining. Models learn by predicting the next word in huge amounts of text, with no true or false labels attached to each statement. Spelling and parentheses follow consistent patterns, so errors there vanish with scale, but arbitrary low-frequency facts, like a pet's birthday, cannot be predicted from patterns alone and hence lead to hallucinations.
The second is how models are graded. The paper argues that training and evaluation procedures reward guessing over acknowledging uncertainty. On a test scored only for accuracy, a guess sometimes scores and "I don't know" never does, so guessing wins. OpenAI illustrates this with its SimpleQA results:
| Metric | gpt-5-thinking-mini | OpenAI o4-mini |
|---|---|---|
| Abstention rate | 52% | 1% |
| Accuracy rate | 22% | 24% |
| Error rate | 26% | 75% |
The figures are from OpenAI's write-up. The older model scores slightly higher on accuracy, and a leaderboard that ranks only accuracy would prefer it, yet it gives a wrong answer nearly three times as often. OpenAI's conclusion is that strategically guessing when uncertain improves accuracy but increases errors and hallucinations.
Do LLMs still hallucinate in 2026?
Yes, less often on many tasks, but it has not gone away. OpenAI wrote in September 2025 that GPT-5 has significantly fewer hallucinations especially when reasoning, but they still occur, and that hallucinations remain a fundamental challenge for all large language models. Its paper also argues that accuracy will never reach 100%, because some real questions are unanswerable, but that hallucinations are not inevitable because language models can abstain when uncertain. That is the lever a system builder can pull.
How can you detect a hallucination?
Three methods work at different levels.
Check the claim against the world. In Mata, the fabrication surfaced because the court conducted its own search for the cited cases but was unable to locate multiple authorities. Any claim with a checkable identifier, such as a case citation, a package name, an order number or a URL, can be verified automatically before a person sees it.
Check the answer against its sources. When a system answers from retrieved documents, you can test whether each statement is supported by them. OWASP's prompt injection guidance suggests evaluating responses with the RAG Triad: context relevance, groundedness, and question and answer relevance.
Check the model against itself. The Nature method samples several possible answers to each question and clusters them into answers that have similar meanings, then measures how spread out the meanings are. High spread signals a likely confabulation. The authors note its limit: it does not help when LLM outputs are systematically bad, so a model that is consistently wrong passes.
How do systems reduce hallucinations?
Separate what the model does from what the system around it does. You cannot retrain the model's incentives, but you control the rest:
| Control | What it prevents | Basis |
|---|---|---|
| Retrieval with citations | Answers about your business invented from general training data | RAG produced more specific, diverse and factual language than a model without retrieval |
| Permission to abstain | Confident guesses when the sources are silent | OpenAI: it is better to indicate uncertainty or ask for clarification than provide confident information that may be incorrect |
| Automatic verification | Fake citations, packages, IDs and links reaching users | Checkable identifiers can be looked up, as the court did in Mata |
| Evaluation that punishes confident errors | Choosing a model or prompt that guesses | OpenAI: penalize confident errors more than you penalize uncertainty |
| Human review where errors are costly | Legal, financial or customer commitments going out unchecked | The court's point that the professional keeps the gatekeeping role |
Two of these deserve emphasis. Abstention has to be designed in: the prompt must say what to do when the sources do not contain the answer, and the evaluation set must include such questions, scored so that "I don't know" beats a wrong answer. And customer-facing output needs the most care, because, as the Air Canada tribunal put it, the company is responsible for all the information on its website, chatbot included.
This is also why many business processes are better served by a workflow with a narrow model step than by an open-ended assistant, as we set out in AI agents vs workflow automation. Qubba builds these systems on a fixed written quote after a 30 minute call, and states the limits up front: AI output needs checks appropriate to the task, and a successful demonstration does not establish accuracy on every future input.
Frequently asked questions
Which LLM hallucinates least?
There is no stable answer, because rates depend on the task, the questions and how abstentions are scored.
OpenAI's own table shows a newer model with lower accuracy but a far lower error rate than an older one, so a ranking by accuracy alone can point the wrong way.
Test candidate models on your own questions and count wrong answers, not only right ones.
Can hallucinations be eliminated completely?
Not by improving accuracy alone.
OpenAI argues accuracy will never reach 100% because some questions are unanswerable, but that hallucinations are avoidable in principle because language models can abstain when uncertain.
In practice, systems reduce them to a level the use case can tolerate and route the rest to a person.
How do I check if an LLM is hallucinating?
Verify any specific, checkable claim against a primary record, as the court did in Mata.
For answers drawn from documents, check that each statement is supported by the retrieved text.
For open questions, ask the same thing several times: the Nature method flags answers whose meaning varies between samples.
Is a hallucination the same as the model lying?
No.
The Nature authors separate confabulations from cases where a model lies in pursuit of a reward, and OpenAI describes hallucination as a statistical error that training and grading reward.
The model is not trying to deceive.
It produces the most plausible answer it can, and plausible is not the same as true.