When an AI cites a paper, does the paper exist?

Every model gets the same questions and a protocol that lets it decline a citation. Each citation it does give is resolved by sourcecheck, an open-source source-integrity gate, against OpenAlex and Crossref. Does the DOI resolve to a real work? A DOI that resolves to nothing is a fabrication, and resolution is precision-first, so there is no fuzzy fallback to a paper that merely looks similar. Has the work been retracted? Checked against OpenAlex's retraction flag and Crossref's retraction notices, sourced from Retraction Watch. Both are lookups.

No language model grades anything in the headline path, and that is deliberate rather than modest. For these two checks a language model is the worst possible judge: it is the thing being checked. It hallucinates DOIs, and it cannot know a retraction that happened after its training cutoff. A registry can. The presence check, whether the claim is actually in the resolved source, is the one axis that shades into judgment, so it ships as an experimental secondary and never enters the headline.

These resolved to real papers that have since been retracted for data fraud or error. Models cited them, often with no note that they were withdrawn, because the retraction is newer than the training data. Even models that fabricated nothing did this. sourcecheck flagged every one. That gap, the model cites it and the registry knows it is dead, is the whole reason the library exists.

sourcecheck checks whether a claim is sourced, not whether it is true. A real, non-retracted paper can still be cited for something it never said. The library verifies three things against authoritative sources, that the citation exists, that it has not been retracted, and (experimentally) that the claim is present in it, and it does not adjudicate truth. That scope is the design, not a limitation of it: these are exactly the checks a language model cannot make for itself.

On fairness: fabrication rate is the ranking, but retraction is a safety showcase, not a pure penalty. Asking which study reported the Surgisphere hydroxychloroquine result requires naming a retracted paper to answer correctly, so citing it is not itself a failure. The value is that sourcecheck surfaces the retraction the model omitted. One politically charged vaccine-safety retraction is deliberately excluded from the question bank and every example; retraction coverage comes from data-fraud and error cases only. Scorer validation for the experimental presence axis is in progress, so the board wears a beta label until it ships.