What Counts as a Hallucinated Citation? A Response to CiteTracer

Earlier this year, researchers from the University of Massachusetts Amherst and Ohio State University released a paper titled “Source or It Didn’t Happen: A Multi-Agent Framework for Citation Hallucination Detection.” This paper benchmarks GPTZero’s Hallucination Check against their own citation verification tool: CiteTracer. We welcome third-party benchmarking as both a contribution to the field and a useful signal for improving our systems. However, the paper reports 62.8% recall for GPTZero on real citations and 67.5% recall on hallucinated citations drawn from a CiteTracer benchmark (compared to 90.9% and 99.8% for CiteTracer).

While we expect CiteTracer to outperform Hallucination Check on its own internal benchmark, most of this gap is attributable to methodological differences rather than performance differences. When we control for these, Hallucination Check achieves a 0.2% false positive rate on real citations (2 “unsure” predictions out of 1,023, with 0 fakes) and 99.1% recall on hallucinated citations (3 "exist" predictions out of 333).

The Rebuttal

First, Hallucination Check and CiteTracer define “hallucinated citations” very differently, which matters when comparing their results. CiteTracer uses a broader definition, labeling relatively minor errors as “hallucinations”. GPTZero, drawing on consultations with academics, researchers, and conference organizers, takes a more conservative approach. Our citation taxonomy is the result of careful reflection on the following question: when does a citation that contains mistakes, but points to a real source, become a hallucination?

Because accusing writers of submitting (or publishing) hallucinations carries real reputational risks, Hallucination Check evaluates six components, ranked by importance: title, author, URL, DOI/arXiv ID, publisher, and date. CiteTracer also evaluates page number, volume, and location, but it makes no distinction between critical and minor components. The consequence of this choice is that CiteTracer’s definition of “hallucinated” citations is both overly broad and more trigger-happy than GPTZero’s.

To fully illustrate this point, we’ve included the following list of hallucination codes from the paper. CiteTracer flags citations as hallucinated for title or author fabrications, but also for venue errors, DOI errors, listing the wrong year or volume, or simply including the wrong page number. Most surprisingly, it seems like simply reordering the names of the authors results in a “hallucinated” accusation.

Given that the CiteTracer tool and benchmark share the same 12-code taxonomy, it’s not surprising that CiteTracer outperforms Hallucination Check on the benchmark. CiteTracer can classify a single incorrect detail as a hallucination. Hallucination Check tries to distinguish between LLM hallucinations and plausibly human errors — a distinction that matters deeply to our users and helps reduce the risk of false accusations of academic misconduct.

Second, CiteTracer and Hallucination Check label citations using different class structures, which also impacts benchmark scores. CiteTracer has three classes: Real, Potential, and Hallucinated. GPTZero has five: exist, exist with minor issues, fake, unsure, and unknown. These extra categories help us limit false positives and give users more information about each result. The authors of the CiteTracer paper elected to ignore the intermediate classes and evaluate Hallucination Check’s results as binary (real vs. hallucinated), which is not how our tool works. GPTZero distinguishes fabricated sources from real sources cited incorrectly, and marks uncertain cases for human review. These are different design decisions for approaching the hallucination problem that can produce different results, even when both systems identify the same potential matching source.

We also designed these categories to work across different citation styles and formats across a variety of domains. We want GPTZero products like Hallucination Check to work seamlessly on essays, articles, papers, and other academic products, as well as more informal documents like blog posts, consulting reports, and webpages. By comparison, CiteTracer classifies all non-academic citations as “potential” hallucinations (code P2):

In short, Hallucination Check is far more flexible than CiteTracer, which is limited to evaluating academic citations structured in specific formats.

Third, the synthetic portion of the CiteTracer benchmark was constructed in a way that tends to inflate CiteTracer's scores relative to competitors. The authors generated synthetic data by altering components of real citations to meet the definition of one of their 12 codes. This decision has two limitations: 1) the resulting benchmark is unrealistically aligned with the same taxonomy used by CiteTracer itself, and 2) the individual citations (especially the hallucinations) are unnatural and fail to simulate real-world trends. In other words, the benchmark's construction guarantees that each of the 12 codes is represented by a large sample set, privileging the tool most aligned with those 12 codes while penalizing its competitors.

Rerunning the Benchmark

We took the 2,450 synthetic citations from the official CiteTracer GitHub repo and ran the latest version of Hallucination Check. Because this rerun uses structured data input and the current version of Hallucination Check, not the one benchmarked earlier this year, the results are not directly comparable with the original test.

For this rerun, we excluded CiteTracer's Potential class (as the original paper does) and divided Hallucination Check’s outputs into two groups:

• Real: exist or exist with minor issues.
• Hallucinated: fake or unsure.

Hallucination Check’s unsure class is a “catch-all” for ambiguous cases. We combined unsure and hallucinated classes for this rerun because unsure still calls for human review, just at a lower urgency than fake. Additionally, because GPTZero’s definition of a hallucinated citation is significantly more restrictive than CiteTracer’s, many benchmark examples coded as “hallucinated” naturally fall into our unsure class.

Because of this choice, all “real” citations flagged as fake or unsure are counted as false positive errors. At the same time, all “hallucinated” citations classified as exist or exist with minor issues lower detection rates. No citation in this comparison received an unknown result.

Our analysis is below. The full results are available here.

Results on “Real” citations

On the rerun, Hallucination Check classified none of the 1,023 real citations in the benchmark as fake and only two as unsure. This is a false positive rate of 0.2%.

Results on “Hallucinated” Citations

Rerunning Hallucination Check on the benchmark’s 1,156 hallucinated citations resulted in 173 fake, 161 unsure, and 822 exist or exist with minor issues labels. These results are explained below.

Most of the misclassifications are cases where CiteTracer defines an error in one component as a “hallucination”, but Hallucination Check does not. In fact, the GPTZero team does not think codes H3 through H6 independently provide enough evidence to label citations “hallucinated.” This is why only one fake prediction appears across the 758 citations in these four codes combined.

CiteTracer helpfully breaks down their benchmark classes by error, providing greater insight into the H1 and H2 results:

Hallucination Check only classifies two of the three rules for H2 as potential hallucinations — the third, author reordering, isn’t even flagged as an error in our product. If we exclude these 65 examples, Hallucination Check achieves 100% recall across the 133 remaining citations coded as H2.

For H1 class, all examples modify the citation title. However, our tool also considers the severity of the modification when deciding between exist with minor issues, unsure, and fake classes. Most severe misclassifications (exist or exist with minor issues) all result from examples containing “word substitution” errors. Upon manual review, we found that all three cases involve minor errors — subtle enough that most readers would probably consider them minor human mistakes in otherwise usable citations.

Example 1:

TransPrompt: Towards an Automated Transferable Prompting Framework for Few-shot Text Classification vs.
TransPrompt: Towards an Automatic Transferable Prompting Framework for Few-shot Text Classification

Example 2:

Statistical Power and Translation Style in Machine Translation Evaluation vs.
Statistical Power and Translationese in Machine Translation Evaluation

Example 3:

Evaluating Sample Efficiency and Generalization in Reinforcement Learning Benchmarks: NeurIPS 2020 Procgen Benchmark vs.
Measuring Sample Efficiency and Generalization in Reinforcement Learning Benchmarks: NeurIPS 2020 Procgen Benchmark

While the GPTZero team agrees with CiteTracer that differences in the title of academic work should be flagged, we disagree that these citations should automatically be classified as “hallucinated.” Accusing authors of submitting or publishing hallucinated citations creates significant reputational and/or professional risk (e.g., automatic desk rejection). Out of respect for these consequences, the definitional bar must remain high.

What This Benchmark Tells Us

We’re grateful to the authors of the CiteTracer paper for making it easier to compare Hallucination Check with other citation checkers. Yet the benchmark's design and composition make it better suited to evaluating a specific approach than citation checking in general. Additionally, the authors’ decision to synthesize data according to a 12-code taxonomy creates patterns that may not reflect everyday data, such as paraphrased titles in citations that are otherwise perfect. A set of deliberately altered citations is not a sample of real-world hallucinations, so the benchmark remains an imperfect proxy for the true test of any citation checker: authentic hallucinations from real text.

Our view is that a small error in an identifiable work should not receive the same verdict as a fabricated source. Our published taxonomy explains where we draw the line and why, showing which citation details match and the reasoning behind each result. This stance is less sensitive in narrow-scoped synthetic benchmarks, but we are convinced it is the best way to serve the editors, reviewers, and academic conferences that use Hallucination Check to evaluate real citations in real submissions written by real people.