An AI hallucination is a confident, fluent answer from a chatbot such as ChatGPT, Gemini or Claude that is false or unsupported, for example an invented citation, a broken link or a wrong date. It happens because language models generate text by predicting likely words rather than looking facts up. To fact-check an AI answer, verify every source, link, number and quotation against an independent original before relying on it.
- Language models predict plausible text; they have no built-in check that what they write is true.
- The most damaging hallucinations are fake references: real-looking authors, journals, DOIs and URLs that do not exist.
- Specific details such as numbers, dates, names and quotations are the parts most likely to be wrong and the easiest to check.
- Asking the model to admit uncertainty and giving it source documents reduces hallucinations but does not remove them.
- Search-grounded tools help with recent facts, but you still need to open the cited page and confirm it says what the answer claims.
Chatbots are useful for drafting, summarising and explaining, and most of what they say on common topics is accurate. The problem is that their wrong answers look exactly like their right ones. This guide explains where hallucinations come from, the forms they usually take and a practical routine for checking an answer.
Why large language models hallucinate
They predict the next token
A large language model (LLM) is trained on a huge body of text to predict the next small piece of text, called a token, given everything before it. Asked a question, it produces the continuation that best fits the patterns it learned. When the training data contained the answer many times, the most likely continuation is usually correct. When it did not, the model still produces the most plausible-looking continuation, because nothing in the basic mechanism says "stop, I don't know".
That is why fabrications are so convincing. A made-up reference has the right shape: realistic author names, a journal that publishes in that field, a sensible year and a DOI in the correct format. The model has learned what citations look like far better than it has memorised which ones exist.
Training rewards answering
After pre-training, models are tuned to be helpful, and both human raters and automated benchmarks have tended to reward a complete answer over "I'm not sure". Researchers at OpenAI argued in a 2025 paper, Why Language Models Hallucinate, that grading which gives no credit for abstaining encourages models to guess. Newer models are better at expressing uncertainty, but the tendency has not gone away.
Training cutoff
A model's knowledge stops at its training cutoff date. Asked about later events, new laws, current prices or the latest software version, a model without access to search may describe an older situation as current or fill the gap with a guess.
No retrieval by default
Unless a chatbot is connected to web search or to documents you provide, it is answering from memory alone. It cannot open the paper it is citing or check that a URL works. Even when it has search, it can misread a page, mix up two sources or attach a correct citation to a claim the source does not make.
Common types of hallucination
| Type | What it looks like | How to check |
|---|---|---|
| Fake citations | A paper, book or court case with plausible authors, title and journal that does not exist, or a real paper credited with findings it does not contain | Search the exact title in a library catalogue or Google Scholar; read the abstract or relevant section |
| Invented DOIs and identifiers | A DOI, ISBN or arXiv ID that does not resolve, or resolves to an unrelated work | Open https://doi.org/ followed by the DOI; look up ISBNs in a library catalogue; open the arXiv ID |
| Invented URLs | A link on a real domain with a believable path that returns "404 Not Found" | Open the link; if it fails, search the site for the content |
| Wrong numbers and dates | Statistics, prices, populations, release dates or version numbers that are slightly or completely wrong | Find the figure in an official or primary source and check its date |
| Misattributed quotes | A real-sounding quotation credited to a famous person who never said it | Search the exact wording in quotation marks and trace it to an original speech, book or interview |
| Confident wrong reasoning | Arithmetic errors, invalid logic or code that calls functions that do not exist, presented step by step | Redo the calculation, run the code, check function names in official documentation |
Fake citations have had real consequences. In the 2023 US case Mata v. Avianca, lawyers were sanctioned after filing a brief containing court decisions that ChatGPT had invented. Similar incidents have since been reported in other courts and in published academic papers.
A step-by-step workflow to fact-check an AI answer
You do not need to check every sentence of every answer. Scale the effort to the stakes: a recipe suggestion needs less scrutiny than a medical, legal or financial claim, or anything you will publish under your name.
- Pull out the checkable claims. List every reference, link, number, date, name and direct quotation. These are where most hallucinations hide and where checking is fastest.
- Resolve every identifier. Open each URL, resolve each DOI at doi.org, look up each ISBN and arXiv ID. An identifier that fails, or leads to a different work, is a red flag for the whole reference.
- Confirm each source says what is claimed. A real paper may still not support the sentence it is attached to. Read the abstract, the relevant section or the quoted passage.
- Check numbers against primary sources. Go to the statistics office, regulator, company filing or original study, not a blog that repeats the figure. Note the date of the number.
- Trace quotations to their origin. Search the exact words in quotation marks. If the only sources are quote collections with no original reference, treat the quote as unverified.
- Re-derive the reasoning. Redo calculations, test code in a real environment and check that each step follows from the last.
- Read laterally. For important claims, look for two independent, reliable sources that agree, rather than relying on one.
- Remove or flag what you cannot verify. If a claim cannot be confirmed, cut it or clearly mark it as unconfirmed.
Our AI Hallucination Checker automates the first steps. Paste an AI answer and it verifies DOIs against Crossref, tests whether links work, checks arXiv IDs and ISBNs, examines reference lines and lists the statistics and factual claims that still need a human check. It tells you which identifiers are real; reading the source to confirm it supports the claim remains your job.
Prompts that reduce hallucinations
How you ask affects how often a model makes things up. These techniques help, although none of them makes an answer reliable on its own.
- Give permission not to know. Models are more likely to admit uncertainty when told it is acceptable.
- Provide the source material. Paste the document, report or data and ask the model to answer only from it. This is the single most effective change.
- Ask for quotations. Requiring a short verbatim quote from the supplied text for each claim makes unsupported statements easier to spot.
- Separate facts from inference. Ask the model to label what is stated in the sources and what is its own reasoning.
- Avoid leading questions. "Which study proved X?" invites the model to produce a study. "Is there evidence for X?" leaves room for "no".
- Ask for fewer, better references. Requests for long reading lists produce more invented entries.
An example prompt that combines several of these:
Answer using only the report pasted below.
For each claim, include a short exact quote from the report that supports it.
If the report does not contain the answer, say "Not stated in the report".
Do not add references or figures that are not in the report.
[paste report here]Asking a model to check its own previous answer can catch some errors, but it can just as easily confirm a fabrication. Self-checking is not a substitute for opening the source.
When search-grounded and retrieval tools help
Many assistants can now search the web or read files you upload before answering. This approach, often called retrieval-augmented generation (RAG), gives the model relevant text to work from instead of relying on memory. It helps most with:
- recent events, prices and versions released after the training cutoff;
- questions about a specific document, contract, policy or dataset you supply;
- niche topics that appeared rarely in training data.
Grounding reduces invented sources because the citations point to pages that were actually retrieved. It does not guarantee accuracy. The search may return a poor or outdated page, the model may summarise it wrongly, or it may cite a real page next to a claim that page does not make. Treat the citations of a grounded answer as a reading list to verify, not as proof.
When to be most careful
- Health, law and money. Errors carry real costs, and rules vary by country and change over time. Use AI to understand the questions, then confirm with official guidance or a qualified professional.
- Anything you will publish or submit. Your name is on it, not the model's. Editors, teachers and courts treat fabricated references as the author's responsibility.
- Niche or local topics. The less a subject was written about online, the less the model has to draw on.
- Very specific requests. Exact page numbers, case citations, statute sections and precise statistics are especially prone to invention.
If you are reviewing text someone else produced and suspect it was generated, our guide on how to tell if text was written by AI covers detection and its limits, and the AI Detector can provide one more signal. Unverifiable references are often stronger evidence than any detector score.
Frequently asked questions
Why does ChatGPT make up sources?
Because it generates text that looks like a likely answer. It has learned the format of citations very well, so when it lacks a real reference it can produce one with plausible authors, titles and identifiers that do not correspond to any actual publication.
How can I check if a reference from AI is real?
Search the exact title in Google Scholar or a library catalogue, resolve the DOI at doi.org, and confirm the authors, journal and year match. Then read enough of the source to confirm it supports the claim. A tool such as our hallucination checker can test DOIs, ISBNs, arXiv IDs and links in bulk.
Do newer AI models still hallucinate?
Yes. Newer models generally hallucinate less on common questions and are better at saying when they are unsure, but every current model can still produce false statements, especially on niche topics, recent events and requests for exact details.
Does web search stop AI hallucinations?
It reduces them, particularly invented sources and outdated facts, but it does not stop them. The model can still misread or misrepresent the pages it found, so open the cited pages and check.
Is a hallucination the same as a lie?
No. A lie implies intent to deceive. A model has no intent; it produces fluent text without a reliable internal check on whether that text is true. The effect on the reader can be the same, which is why verification matters.