AI citation checker: is the source real, and does it say that?

How the AI citation checker works

First, a language model reads your text and lists two things: the entries in the reference list, and every sentence that cites one of them, matched to its entry. It writes nothing and judges nothing; author-year citations, numbered ones and footnotes all work.

Second, each reference is looked up by its title, in OpenAlex first, then Crossref, then Semantic Scholar. A reference counts as found when a work with that title exists; the year and first author are then compared with what your list says, allowing a year either side for the gap between online and print publication. Where an index holds no abstract, the checker asks Europe PMC, which carries abstracts publishers withhold elsewhere.

Third, Jev, a judgment model, reads each citing sentence together with the abstract of the work it cites and picks one of four verdicts. It sees no other part of your text, so a claim is judged on what the source reports, not on how persuasive the paragraph around it is.

Not found: a fake reference, or a gap in the index?

Fabricated references are the best-known failure of chatbots used for research. In one study, ChatGPT wrote short literature reviews on 42 topics; 55% of the GPT-3.5 references and 18% of the GPT-4 references described works that do not exist, and many real ones carried substantive errors (Walters & Wilder, 2023). A fabricated reference usually looks right: plausible authors, a journal that exists, a title in the field's style.

That is why a reference the AI citation checker cannot find deserves a closer look before anything else. Not found does not prove fake, though. Books, government reports, theses and conference papers are thinner in these indexes than journal articles, and a paper published last month may not be listed yet. Search the exact title in your library catalogue or Google Scholar; if it is nowhere, treat it as invented.

Details differ is the quieter problem: a real work listed under the wrong year or the wrong first author. It is the kind of error that makes a reader doubt the rest of the list, and it is easy to fix once you know which entry it is.

Supports, partly, not addressed, contradicts

Each citing sentence gets one of four verdicts, with the probability behind it:

• Supports. The source reports this finding in substance; the wording may differ.
• Partly. The source backs only part of the claim or a weaker, more general version of it: a specific figure, group, place or effect in the claim is not in the source.
• Not addressed. The source is not about what the claim says.
• Contradicts. The source reports the opposite of the claim.

Partly is the verdict to read most carefully. It usually means the source is on your topic but your sentence adds something it does not report: a number, a population, a cause. Jev is asked twice with the verdicts in opposite orders, because a judgment model can lean towards whichever option comes later on a close call; when the two orders disagree, the report says close call instead of picking one.

Example: four references, three planted problems

The example in the tool is a paragraph on later school start times, written for this page with three planted problems: a study that does not exist, a real study listed under the wrong year, and a real review cited for a result it never reports. This is the checker's real output on 2026-10-03, in 8.8 seconds:

All three problems were caught. The invented study was found in none of the indexes. The crash study was found, published in 2008, not 2011 as listed. And the review was marked partly, not supports, when cited for a 20 percent rise in test scores: its abstract is about sleep and calls for better measures of academic performance, but reports no such figure. The same review, cited for the finding it does report, longer sleep, was marked supports.

What the citation checker does not check

It reads abstracts, not full papers, so a claim drawn from deep in a paper's results can come back as partly or not addressed even when it is right; open the source and check the page you used. It does not check direct quotes, page numbers or formatting, and it does not look up web pages, legal citations or works without a title of three words or more. The checker gives you the list of references to look at first; the decision about each stays with you.

Teachers can use it the same way on a submitted paper: the references marked not found or partly are where a conversation with the student starts. For grading the writing itself, the essay grader scores an essay on six criteria.

Questions