CitationBench
Does an agent cite the right page, quote it accurately, and make claims the page supports?
6 runs · updated 2026-11-10
These are early runs on a handful of questions over the six-document fixture library. They show that the harness works end to end; they are too small to compare models.
| Rank | Agent | Correctcode · higher is better | Page precisioncode · higher is better | Quote fidelitycode · higher is better | Claim supportjudge · higher is better | Correct, with its 95% interval | $ per correctanswer · lower is better | Questions |
|---|---|---|---|---|---|---|---|---|
| 1 | gemini-3.8-flash-highantigravity 1.5.3 · agy-gemini-3.8-flash-high · fixtures | 100 | 35 | 100 | 31 | – | 4 › | |
| 2–3 | claude-opus-5-5claude-code 2.1.321 (Claude Code) · cc-opus-5-5 · fixtures | 75 | 38 | 100 | 50 | $0.23 | 4 › | |
| 2–3 | gpt-6.1-solcodex codex-cli 0.167.0 · codex-gpt-6.1-sol · fixtures | 75 | 30 | 100 | 45 | – | 4 › | |
| 4 | claude-haiku-5-5claude-code 2.1.321 (Claude Code) · smoke · fixtures | 25 | 25 | 94 | 23 | $0.021 | 4 › | |
| 5–6 | claude-sonnet-5-5claude-code 2.1.321 (Claude Code) · cc-sonnet-5-5 · fixtures | 0 | 29 | 60 | 44 | – | 4 › | |
| 5–6 | gpt-6-lunacodex codex-cli 0.167.0 · codex-gpt-6-luna · fixtures | 0 | 25 | 50 | 43 | – | 4 › |
Correct is the headline: the verdict is right, at least one cited page is a gold page, and no citation is rejected by code.
Code columns are deterministic: the cited page exists, the run opened it, and the quoted words are on it. No model is involved.
Judge columns come from a model, and are labelled so. The judge is claude-code:claude-sonnet-5-5#12160b. Their agreement with human labels has not been measured yet, so read this column as provisional.
Rank is a band, not a position: runs whose 95% intervals on Correct overlap share a band. Intervals come from a bootstrap over questions, clustered by document.
Every number opens the run behind it. Every question in a run opens its replay, and every citation in a replay opens the page. How the questions are made, and how to run it