Hallucinations in AI are more than a rare quirk: in Stanford’s 2026 AI Index, hallucination rates across 26 top AI models ranged from 22% to as high as 94%. These are becoming a serious research integrity issue. In May 2026, arXiv announced that authors who submit papers with unchecked AI-generated content, including hallucinated citations, could face a one-year ban. 

This followed rising concern about fake citations in academic work, including our finding of over 50 hallucinated citations in NeurIPS 2025 papers. We also recently chased down every citation in an Ernst & Young (EY) Canada cybersecurity report and found most were hallucinated – the FT covered the story of how EY subsequently retracted the study. 

We know that while many educators already use AI detection, citation hallucinations are less familiar. After reading this guide, you’ll know how to spot fake citations, verify AI-generated claims, and use GPTZero’s Hallucination Detector.

TL;DR: 

AI hallucinations are false or unsupported outputs that can look incredibly convincing, and the most common academic version is a fake citation. Checking for hallucinations starts with examining the title, authors, journal, DOI (Digital Object Identifier), URL, date, and whether the source actually supports the claim. GPTZero’s Hallucination Detector can help detect suspicious citations, although human review should make the final decision.

What Are AI Hallucinations?

According to IBM, “AI hallucinations are when a large language model (LLM) perceives patterns or objects that are nonexistent, creating nonsensical or inaccurate outputs.” 

One of the most common is a fake citation, where the AI completely makes up a source like a paper or case - or URL - that doesn’t actually exist. These hallucinated citations are especially difficult because many of them pass what GPTZero’s Senior Machine Learning Engineer Nazar Shmatko calls the “vibe citation” test: they look real enough at first glance. 

They may have an academic-sounding title, plausible authors, a journal name, and even a DOI-style structure. But when you actually check them, the source is nowhere to be found, or the citation may point to something different from what the AI claims.

As Shmatko and GPTZero’s Academic Writing Editor Paul Esau explain in GPTZero’s report on Hallucination Check, these citations “seem plausible on first glance and require high levels of technical expertise or time-intensive research to identify.” 

Similarly, there can be false facts or statistics, or a fabricated quote. An AI tool may attribute a quote to a real person, university, government body, or research organisation, even though the quote does not appear in the original source – this can be tricky to track down, as it might actually sound like something they could say. 

Finally, hallucinations can appear as overconfident summaries, and present a controversial issue as if there is a clean and simple answer. In these cases, the hallucination goes beyond being a single fake fact and instead represents the fake certainty of the conclusion. 

What AI Hallucinations Look Like 

As outlined in our analysis of papers accepted by NeurIPS 2025 with confirmed hallucinations, the following table shows the difference between a real citation, a flawed citation, and a hallucinated citation according to our methodology. The differences are highlighted in red.

How to Check for AI Hallucinations

As one professional in the GPTZero community told us, “the most important drawback of AI is to hallucinate.” He described that when a document contains “too many sources” and there is “not enough time to check all of it,” he needs a faster checking method, although he admitted that even hallucination tools need to be tested (as opposed to blindly trusted). 

This is where GPTZero’s Hallucination Check comes in – it exists to detect potentially misleading claims in text and give recommendations for sources that support or contradict those claims. Our tool allows you to find any arguments or “claims” in a document that may require more scrutiny, and then links to helpful sources to dive deeper into your analysis and provide helpful context. You can pull these into your own research, or share your results to improve someone else's.

Essentially, it looks at citations in a more structured way than a simple search. As our technical report explains, the system compares the key parts of a citation (including the title, authors, publisher or journal, publication date, URL, and DOI) against possible source matches. One matching detail isn’t sufficient, as a fake citation could include a real author but a fake title, or a real journal but a fake DOI.

GPTZero’s Hallucination Check will detect as many checkable, objective claims in your text and match those to sources from online, academic, and publicly available data coming from AI-powered search engines. You may find sources that would directly support or contradict these claims. You can cite the relevant snippet from the source, and also citations in MLA, Chicago, APA, BibTeX and IEEE. 

To be extra sure of the validity of citations, you can use the same principle manually. When checking a citation that doesn’t quite look right, ask whether these components match:

• Does the title match? 
• Do the authors match? 
• Does the journal exist? 
• Does the DOI resolve? 
• Does the URL point to the right source? 
• Does the publication date make sense? 

Why AI Hallucinations Happen

Instead of presenting verified truth, AI predicts likely language: In this sense, it may simply not know when information is missing, and can combine real details into false combinations. As AI is trained to give fluent answers, its confident tone can often mask the fact that it doesn’t actually always know exactly what it is talking about. 

MIT Sloan Technology Services (STS) explains why generative AI can be inaccurate:

• It learns from a flawed internet: These models are trained on huge amounts of online text, including legitimate research, random blog posts, misinformation, and all the usual human biases. But the model doesn’t “know” which is which; it just absorbs patterns in a massive fruit salad of information. So if the training data contains falsehoods or bias (and it does), the model will happily reproduce them.
• It’s built to sound plausible, which isn’t the same as being right. Under the hood, a generative model is an extremely powerful autocomplete: its job is to predict the next likely word, which differs from sticking to the facts or getting things right. 
• It has no built-in sense of truth. These systems don’t have a mechanism for separating “true” from “false”, and even if you trained a model only on accurate information, it would still remix that material in new ways (and some of those combinations would inevitably be incorrect). 

As Shmatko explains, “AI hallucinates when the information required to fulfill a request is missing, ambiguous, or outside of the training dataset. The model may confidently fill-in the gaps with plausible-sounding guesses. This tendency is amplified by the pressure to always provide a complete answer (just try to shush any voice assistant ).” 

“As a result, when a user asks an LLM to generate a citation without sufficient context, the model may make up sources. Such misinformation can go unnoticed because AI is optimized to produce answers that sound coherent.”

How to Reduce the Risk of AI Hallucinations

Verifying citations is a multi-step process: identify the citation, see whether it can be checked, find possible matches, compare the components, and then classify the result. This is the same multi-layer process that goes into the thinking behind GPTZero’s Hallucination Check. Similarly, human reviewers should do the same thing: check whether the source actually exists, and if it actually backs up the claim being made. 

The danger of hallucinations is that they can also affect the broader knowledge ecosystem, and slowly dilute the very definition of accuracy. As GPTZero’s Head of Machine Learning Alex Adam says, “Even if a paper using hallucinated citations isn't ‘slop’, it can have negative impacts on the research community. Claims made in the paper relying on non-existing sources will be used to inform future works, misleading other researchers.” 

Specifically, there is a risk that those researchers then go on to continue misleading other researchers, and so on. Adam explains, “‘An excerpt in a paper like "Jane Smith et al. established that…’ is harmful because the reader will take it at face value, leaving out the skepticism they might have for obvious slop.”

GPTZero’s AI detector helps identify likely AI-generated writing, while Hallucination Detector focuses on whether citations and sources are real and correctly represented. Together, they help reviewers evaluate both authorship signals and source reliability.

Try GPTZero’s Hallucination Detector to validate sources in seconds.

How Are Teachers Using GPTZero’s Hallucination Detector?

While standard AI text detection remains the primary focus for most educators, teachers also use the Hallucination Detector to make sure a student’s research trail is legitimate. Teachers generally use the Hallucination Detector for major assignments like theses and extensive research reports. 

They use it for:

1. Catching fake sources and DOIs: Educators mainly use the tool to identify fake references invented by generative AI. Since AI models often hallucinate non-existent article titles or generate random, inaccurate DOI numbers, the detector helps teachers flag these citations by confirming that the document cannot actually be found on the internet. 

As one grader said, “You see, for a citation, sometimes when you generate work with AI, it will give you a DOI number for the article. But sometimes AI tools will produce a random number instead of the actual DOI number — the document identification number. So the system is able to tell whether it is AI-generated because it probably checks whether it can find that document on the internet, and if the DOI number is wrong, it can’t find it.”

2. Evaluating claim support: Some instructors use the tool to check how well a student is actually interacting with their cited material. The scanner can provide feedback indicating if a student has simply dropped a stated fact into their paper without actually expounding upon, critiquing, or properly discussing the idea to advance their own argument.

As another grader told us, “I’ve gotten feedback that sometimes the citations have not been done well, because it is able to tell me that the person has just stated whatever is written in the text, but has not expounded, discussed, or critiqued the work. Because you see, when you are doing article writing, you have to cite the work, critique it, discuss it, and explain it so that it can align with your argument.”

3. Testing the Tool for Personal Understanding: As the technology is still relatively new, some educators use the hallucination detector to scan their own authentic writing to test its validity.

Conclusion

While AI hallucinations might have started as a quirk of LLMs, they’ve now become a major integrity issue, especially when a source that doesn’t actually exist makes its way into a professional and official knowledge base. Checking for hallucinations has to become part of the research process itself, and GPTZero’s Hallucination Detector can help. As AI-generated text becomes more common, it becomes even more important to triple-check the quality of the information being presented. 

Use our reliable source checker and AI reference finder to quickly find sources from text, essays, or research papers.

FAQs

Why does ChatGPT hallucinate?

ChatGPT generates likely language patterns rather than verifying every fact, and so might make up an answer that sounds fluent even when the underlying claim is false.

How do you stop AI from hallucinating?

You cannot completely stop AI from hallucinating, but you can reduce the risk by using trusted source material and asking the model to show uncertainty. 

What tools detect AI hallucinations?

GPTZero’s Hallucination Detector can help flag fake citations. Ideally, the tool should be matched with manual source checking where possible. 

What are the implications of AI hallucinations?

AI hallucinations can lead to flawed research and in education, they also create academic integrity risks if students do not take the time to verify their own citations.