Why AI Citation Checkers Get Fooled by Fake DOIs (And How We Fixed Ours)

TL;DR: An AI citation checker's whole job is to tell you when a reference is fake. We found a case where ours — and, by the underlying mechanism, many verification approaches built the same way — could be tricked into doing the opposite: confidently verifying the real 2017 Transformer paper, "Attention Is All You Need," against a fabricated duplicate registered under a byte-identical title and the same eight authors, dated November 2025, through a publisher with no connection to machine learning. The fake entry was indexed in two separate databases, and in at least one plain title search it ranked above the real paper. This post walks through how that duplicate DOI fooled a naive title match, what in our own verification logic let it happen, and the fix we shipped: matching against the parsed title instead of scanning raw citation text, retrying on a low-confidence match instead of only on zero results, and letting corroborating sources outvote an isolated bad one.

The problem: "verified" doesn't always mean what you think

A common way to build an AI citation checker is to take a reference, query it against academic databases like Crossref, OpenAlex, or Semantic Scholar, and if something comes back that looks close enough, mark it verified. That sounds solid — until you realize those databases aren't curated by hand. Anyone can register a DOI. Anyone can get a paper indexed. And increasingly, someone is doing exactly that at scale, targeting the most-cited papers in a field because a citation to a "real-looking" DOI is worth more than an obviously fake one.

We found this the hard way, testing AccuraCite against one of the most cited papers in machine learning: "Attention Is All You Need" (Vaswani et al., 2017) — the paper that introduced the Transformer architecture.

Case study: a 2017 paper that "became" a 2025 paper

We ran the real citation through AccuraCite's full verification pipeline, which cross-checks against seven sources at once (OpenAlex, Crossref, Semantic Scholar, PubMed, DBLP, arXiv, and CORE). The result:

status: mismatch
source: Crossref
publisher: Shenzhen Medical Academy of Research and Translation
year: 2025
DOI: 10.65215/ctdc8e75

That's wrong on every count. The real paper is from 2017, published via NeurIPS/arXiv, with no connection whatsoever to a medical academy. So what happened?

We traced it to a duplicate DOI — 10.65215/ctdc8e75 — registered under a byte-identical title, with the exact same eight authors copied from the real paper, dated November 2025. It wasn't a coincidence or a parsing error: it's a fabricated, paper-mill-style entry designed to look exactly like the real thing to any system that matches on title text alone. And it wasn't just in Crossref — the same duplicate was indexed in OpenAlex too, in some cases ranking above the real 2017 paper in a plain title search.

Verify this yourself

We're not asking you to take our word for it. Crossref's public API record for that DOI is permanent and independent of any single website staying online:

curl https://api.crossref.org/works/10.65215/ctdc8e75

That returns "publisher": "Shenzhen Medical Academy of Research and Translation", "posted": {"date-parts": [[2025, 8, 23]]}, "type": "posted-content" — and an abstract that's a word-for-word copy of the real Transformer paper's abstract, BLEU scores and all.

The DOI itself resolves through a redirect chain worth seeing:

doi.org/10.65215/ctdc8e75
  → langtaosha.org.cn/index.php/lts/preprint/view/10   (302)
  → langtaosha.org.cn/lts/en/preprint/view/10           (404 — gone)

By the time we checked, the actual landing page had already been taken down. That's not a hole in this story — it's consistent with it. Paper mills like this rotate and remove entries; the DOI registration and its metadata persist in Crossref regardless of whether the page behind it stays up, which is exactly why checking the database record directly (not just clicking a link and seeing if it loads) matters.

Here's what actually saved us: AccuraCite doesn't trust a single source. It queries all seven providers concurrently and looks for agreement.

Once DBLP, arXiv, and Semantic Scholar all independently agreed on the real 2017 paper, that consensus outweighed the two providers carrying the fake entry — and the citation correctly came back verified, year 2017, no mismatches.

But getting there took a real fix, not luck.

Why this keeps happening

Two separate things have to go wrong for a fake entry to win:

The data itself is poisoned. Academic databases aggregate what publishers submit; they don't independently fact-check every DOI registration. A byte-identical title with copied author names is enough to pass a naive title match.
Verification tools can give up too early. We found that our own fallback logic only retried with a cleaner search query when a provider returned zero results — a limitation that isn't unique to our approach. In practice, a noisy first search (mixing author names, year, and venue into one query) rarely returns nothing; it returns something irrelevant instead, and a tool that stops there either misses the real paper entirely or, worse, accepts whatever came back first.

That second point is the one within a tool's control, and it's the one we fixed.

How we fixed it in AccuraCite

Three concrete changes, all shipped and re-verified against live APIs:

Fallback logic now triggers on a bad match, not just zero matches. If the best result from a provider's initial search doesn't actually score well against your citation's title, we now retry with a clean, title-only query instead of accepting a low-confidence result.
Title matching compares against your parsed title first, not a scan of the whole pasted citation text. The old approach could be fooled by a short candidate title happening to appear as a substring inside a longer citation block — a fabricated reference containing a few words in common with something real could slip past the fuzzy-match threshold. Matching against the actual extracted title closes that gap.
Cross-provider consensus does the real work. No single source is trusted blindly. When providers disagree, the citation with more independent corroboration wins — which is exactly what let three legitimate sources overrule two poisoned ones in the case above.

Read more about how the full pipeline works on our How It Works page, or try it yourself on the AI citation checker.

What this means if you're using AI to draft citations

If you're using ChatGPT, Claude, or any LLM to help draft a literature review or reference list, two things are true at once: the model can hallucinate citations that don't exist at all, and the citations it gets "right" can still be quietly wrong in ways a single-source check won't catch. A tool that says "verified" after checking one database isn't necessarily wrong — but it isn't giving you the whole picture either.

The practical takeaway: don't treat any single green checkmark as proof. Cross-reference against more than one source, and be specifically skeptical of citations to very recent DOIs on very old, famous papers — that mismatch is one of the clearest signals of exactly this kind of duplicate-DOI paper mill.

FAQ

Is this a problem with AccuraCite specifically, or citation databases in general? The fake DOI entries live in the underlying databases (Crossref, OpenAlex) themselves, not in AccuraCite — those are aggregators, not fact-checkers. The fix is in how a verification tool uses that data: single-source trust is fragile, cross-provider consensus is much harder to fool.

How common are these duplicate-DOI paper mills? We found this specific case while testing one of the most famous papers in machine learning, which suggests it's not an isolated incident — high-citation-count papers are the most attractive targets for this kind of scheme, since a citation "hit" is worth more the more well-known the original is.

Does AccuraCite catch this every time now? The consensus mechanism means a fake entry needs to outnumber genuine corroborating sources to win, which is a much higher bar than fooling a single-source checker. It's not an absolute guarantee — no automated tool can be — but it's a meaningfully harder target than a single-source check.