The link works. The sentence isn't in it.
The citation resolves and the claim it supports is not in the document. What per-claim attribution looks like, worked through one CFTC enforcement figure.
Updated July 31, 2026
You clicked the citation before sending the memo. The page loaded, the domain was the right one, and the sentence you had written was nowhere in it.
I know that moment from the inside, because this site published it. Our worked run put USD 135,000,000 in the penalty cell for Vitol and hung CFTC release 8326-20 underneath it. That release resolves. It is the right regulator, the right firm, the right date. Open it and the figure it carries is USD 95.7 million, the CFTC’s own order. The 135,000,000 came out of the Justice Department resolution and is not in that release.
The topic matched and the document did not, on the field an editor challenges first. A link that resolves proves a document exists. Whether it says what your sentence says is a separate question, and it is the one that matters.
Take the number this post is built around. The USD 1,186,345,850 the CFTC ordered in its 2022 Glencore case is correct. It is also a penalty added to a disgorgement, it covers three companies, and it is partly offset against payments to the Justice Department. A source list at the foot of a report has nowhere to record any of that.
Per-claim attribution means a source attached to a single value. This post works through what that catches and what it costs, using one enforcement figure. It assumes you already use a deep research mode, and it stays off retrieval quality and search coverage.
Link validity above 94%, factual accuracy between 39% and 77%
A May 2026 paper benchmarked source attribution across fourteen closed-source and open-source language models. Among the strongest frontier models, link validity stayed above 94% and topical relevance above 80%. Factual accuracy reached only 39% to 77%. Fewer than half of the open-source models managed to produce a cited report at all in a one-shot setting (Cited but Not Verified, Onweller and colleagues, 7 May 2026).
The links work. The sentences, often enough, are not in them. And the second finding in that paper is the one that should change how these systems get built: fact-check accuracy dropped by approximately 42% on average across two frontier models as tool calls scaled from 2 to 150. Searching harder made the attribution worse.
One CFTC release settles three things about one number
LoQuery uses four words for a single value in a table, and the rest of the post leans on them:
- Cell. One item and one field. One row, one column.
- Verdict. One of four labels on that single value: confirmed, confirmed with caveats, uncertain, mismatch.
- Score. The confidence behind the label, from 0 to 1.
- Caveat. A named fault from a fixed list, not written fresh each time.
Below is one cell from a run profiling commodities-trading enforcement actions. An analyst asked for the regulator, the penalty amount, the date of the order and the primary filing, for three firms.
- Item
- Glencore Ltd
- Field
- Penalty amount
- Value
- USD 1,186,345,850, penalty plus disgorgement, three entities collectively
- Verdict
- confirmed with caveats 0.78
- Caveats
- entity_scope_broader_than_item
- Why
- Score 0.78; the order settles with three entities collectively defined as Glencore, so the figure is not attributable to the named item alone.
Cell values, sources and caveats here are real and open to checking. The scores, attempt counts and source totals show the shape of a run and get replaced once a captured session record lands.
The link under that value does not point at cftc.gov, and it does not point at whichever CFTC release covers the case. It points at CFTC release 8534-22, the one document this figure came out of. The second distinction is the one our own table failed, twice: the Vitol penalty above, and the Freepoint cell beside it, where USD 98,551,150 of Justice Department penalty and forfeiture sat over a CFTC order. Both cells now cite the Justice Department releases the figures came from. Open 8534-22 and three things are settled inside a minute.
Two kinds of money, added together. The release reads: “Glencore is required to pay a total of $1.186 billion, which consists of the highest civil monetary penalty ($865,630,784) and highest disgorgement amount ($320,715,066) in any CFTC case.” A note that calls the whole of it a fine is wrong about USD 320,715,066 of it.
Glencore is three companies. The order settles charges against “Glencore International A.G. of Switzerland, Glencore Ltd. of New York, and Chemoil Corporation of New York (collectively, Glencore)”. The row asked about Glencore Ltd. One of the three is not called Glencore at all. Attributing the total to the named subsidiary is a claim the document does not make, and the caveat on the cell says so.
Payments to the Justice Department are offset against it. “The CFTC order recognizes and offsets certain forfeiture and penalty payments to be made to the DOJ in those cases.” Sum the two regulators’ headline numbers and you have published a total that was never owed by anyone. Our own example data did exactly that on two rows, showing a Justice Department figure and a CFTC figure side by side as though they added, until an earlier audit caught it. That fault was in the arithmetic. The Vitol and Freepoint faults were in the citations. Both kinds got through the same review, which is why I stopped treating a working link as reassurance.
Cell records hold value, source, verdict and caveat together
A bibliography and a cell record differ in one structural way, easiest to see as data. That run touched 19 sources across 5 domains. A list of those URLs holds this much about the penalty figure:
{
"claim": "The CFTC ordered Glencore to pay USD 1,186,345,850.",
"sources": [
"https://www.cftc.gov/PressRoom/PressReleases/8534-22"
]
}Every part of that is true, and not one of the three findings above fits anywhere in it. There is a slot per document and no slot for the edge between a document and a sentence.
A cell record gives that edge somewhere to live. A session record holds the same cell with these keys, per field, per item:
{
"item": "Glencore Ltd",
"field": "Penalty amount",
"value": "USD 1,186,345,850, penalty plus disgorgement, three entities collectively",
"source_url": "https://www.cftc.gov/PressRoom/PressReleases/8534-22",
"verdict": "confirmed_with_caveats",
"score": 0.78,
"caveats": ["entity_scope_broader_than_item"],
"verdict_source": "evaluator"
}Filter that structure to what carries a caveat and this run returns three items out of three. Sort it by score and the weakest surfaces first, at 0.74. Neither operation exists on a list of URLs.
Three firms, twelve cells, no clean row
Here is the rest of that run. Three firms, four fields, twelve cells, each carrying its own deep link.
| Item name | Score | Verdict | Why | Caveats | Regulator | Penalty amount | Date of order | Primary filing |
|---|---|---|---|---|---|---|---|---|
| Vitol Inc 2 attempts | 0.79 | confirmed with caveats | Score 0.79; the USD 135m is a combined DOJ and Brazil resolution, and a parallel CFTC order the same day partly offsets against it. The figures are neither one number nor two that add up. | parallel_action_unreported | US Department of Justice justice.gov/…/vitol-inc-agrees-pay-over-135-million | USD 135,000,000, a combined DOJ and Brazil resolution justice.gov/…/vitol-inc-agrees-pay-over-135-million | 3 December 2020 justice.gov/…/vitol-inc-agrees-pay-over-135-million | Deferred prosecution agreement (FCPA) justice.gov/…/vitol-inc-agrees-pay-over-135-million |
| Glencore Ltd 3 attempts | 0.78 | confirmed with caveats | Score 0.78; the order settles with three entities collectively defined as Glencore, so the figure is not attributable to the named item alone. | entity_scope_broader_than_item | CFTC cftc.gov/PressRoom/PressReleases/8534-22 | USD 1,186,345,850, penalty plus disgorgement, three entities collectively cftc.gov/PressRoom/PressReleases/8534-22 | 24 May 2022 cftc.gov/PressRoom/PressReleases/8534-22 | CFTC order cftc.gov/PressRoom/PressReleases/8534-22 |
| Freepoint Commodities 3 attempts | 0.74 | confirmed with caveats | Score 0.74; a parallel CFTC order charges the same conduct under a different statute and is largely offset against this one, so neither figure alone is the answer and adding them is wrong. | parallel_action_unreported | US Department of Justice justice.gov/…/commodities-trading-company-98m | USD 98,551,150, a DOJ penalty and forfeiture justice.gov/…/commodities-trading-company-98m | 14 December 2023 justice.gov/…/commodities-trading-company-98m | Three-year deferred prosecution agreement, District of Connecticut justice.gov/…/commodities-trading-company-98m |
Sources here are real; scores and attempt counts are illustrative until a session capture lands.
No row came back clean, and I want to be clear that this is the finding rather than the failure. In every case the headline number belongs to more entities, or to more actions, than the name in the row. Three firms out of three. A tool that handed back three tidy numbers would have been confidently wrong three times, and it would have felt more finished than this table does.
The clean-looking answer and the correct answer were different in every row of this run.
Typed steps keep the source attached to the value
One long agent trajectory has room to drift. By tool call 150 the sentence being drafted is many steps from the page that justified it, and the connection is being rebuilt from a summary of a summary. Drift is a plausible mechanism for the 42% degradation above, though the paper does not decompose it, and we do not know how much is retrieval loss versus summarisation loss.
LoQuery decomposes a run into typed steps for that reason. Extract this field, for this item, from this document. Score that value against that document. Judge it in a separate call that sees the value and the page and little else. The link reaches the output because it was never separated from the value along the way.
The design follows from the failure, not from a measurement of the fix. We have not run that benchmark against our own pipeline.
Rejected alternatives: bibliographies, footnote markers, self-grading
Three cheaper shapes were available, and each one stops short of the edge that matters:
| Shape | What it records | Where it stops |
|---|---|---|
| Report-level bibliography | Every document the run touched, once. | No edge from a document to a sentence. |
| Footnote marker in generated prose | One sentence, one document. | Points at the document, not the line in it. |
| Self-grading in the extraction call | A number from the step that wrote the value. | The step is grading its own work. |
A self-assessed number tracks fluency more closely than evidence, which is why the grader never sees the working.
A source list also cannot catch a leading question. Ask a contested question as a believer, then as a skeptic, and two confident answers can come back with real documents under each. What differs is the sentence each document was made to support. We have not measured how often that happens.
Restricting a run to primary sources does not fix this either. CFTC release 8534-22 is the primary document, and the overreach still happened. An authoritative source raises the floor on existence and does nothing for support.
Citation checking and source checking are two different jobs
Two jobs travel under one word. Checking a citation can mean confirming the reference is real and well formed, which a resolving URL and a matching author, title and date settle between them. Checking a source means opening the document and finding the sentence the claim rests on. Only the second one catches the Vitol cell above, where everything about the citation was right except the figure, which came out of a different resolution entirely.
Our only clean row cited a page that did not hold the records
The other worked run on this site profiles three aviation incidents for a Grade 11 class, and we made the same mistake there against a national archive. The row for the 1967 Shag Harbour incident asked for the primary source document, and the cell we published named the RCMP and Defence records held at Library and Archives Canada, citing a curated LAC research list. The list is real. It is maintained by the archive itself, it is genuinely about that incident, and it opens in a browser.
Its archival items are RG77, the National Research Council. There is no RG18 and no RG24 on the page, which is to say no RCMP file and no Department of National Defence file. The only RCMP on it belongs to the curator’s own description of the case: “Canada’s most famous UFO incident, investigated by the RCMP and Canadian Forces.” That is a sentence about the event. We read it as a sentence about the holdings, which is the substitution this whole post is about, made by us, on the kind of source that gets questioned least.
The cell now says what the page says, which is a research list whose archival items are National Research Council records. It reads confirmed with caveats at 0.82, and it was the only row on this site that carried no caveat at all. Both worked runs now show zero in the confirmed-clean chip. A clean row shows the tool agreeing with the question; a caveated one shows what it found when it went and looked.
Worth admitting separately: that cell had been corrected once before, and the overstatement came back. Nobody reintroduced it on purpose. It came back because the page looks like the record, and reading it as the record is the path of least resistance every time somebody touches the file.
What we missed, and what we have not measured
The run flagged the entity scope and missed the offset to the Justice Department. A person reading the release caught that one. The gap in the evaluator is real, and pretending otherwise in a post about unsupported claims would be absurd.
Count the faults in this post and one of them came from the machine. The entity-scope caveat on the Glencore cell is the run’s own work. The offset, the two miscited penalty cells and the archive list were all caught by a person opening a document and reading it against the sentence above it. That audit is a human process and it is not in the product.
LoQuery has not launched. There is no pilot, no paying user, and no automated evaluation of our own attribution against a benchmark like the one above. What is real and what is illustrative in the tables above is marked under each of them. The mechanism is built and running. The score for it does not exist yet, and this post is not the place it gets invented.
Check the largest number in the memo you are about to send
Find the sentence it came from, in the document it came from. If that takes longer than a minute, the citation under it was doing the existence job only, and the memo is carrying a claim nobody has checked. That check costs one minute. I can tell you from our own audit that skipping it costs more.
Four objections, briefly
Isn't per-claim attribution just footnotes?
A footnote points a sentence at a document. A cell record also carries a verdict on whether the document supports the value, a confidence score, and a typed caveat. The footnote records that a source exists; the cell records what checking it found.
Doesn't restricting to primary sources solve this?
No. The Glencore release is the primary document and the overreach still happened, because the figure covers three entities and the question named one. A national archive page on this site failed the same way, holding one agency's records under a claim about another's. Authority raises the floor on existence and does nothing for support.
Can I get this discipline without LoQuery?
The manual version is real: for every load-bearing number, find the exact sentence in the exact document and note what the document scopes it to. It does not scale past a handful of claims, which is the problem we are building against, but for one memo it works today.
Is this an AI fact checker?
No, and the difference is worth stating. A fact checker returns a verdict on a claim: true, false, misleading. A cell record is narrower than that. It holds the document a value came from and a verdict on how well that document supports it, which is a judgment about the fit between a sentence and a page rather than about the world.
The longer argument for defensible output lives on the researchers’ page. Early access is open on the waitlist.