Full Fact, the UK fact-checking charity, spent ten weeks this summer asking five chatbots the same 22 civic questions every day. The results, published this month, were about what you'd expect: often right, sometimes stale, sometimes miscounted, and shifting from one day to the next. The line that matters came from its chief executive. The tone, he said, stays exactly as confident whether the answer is correct, outdated, or about something that hasn't happened yet.
Issue 001 explained why. A model is trained to produce text that reads like an expert wrote it, and that voice is on every answer by design. This issue takes the next step. If an AI's confidence is the least reliable signal you have, what are the more reliable ones, and how do you rank them?
Start with the uncomfortable part. Confidence isn't a weak signal. It's not a signal at all, and it's worse than nothing, because it does one thing very well: it switches off your urge to check. In September, CNN reported, citing four unnamed sources, that an analyst at a US special operations command had used a chatbot to combine shipping data with classified intelligence about a Chinese-flagged ship. The chatbot concluded the vessel was carrying components for a nuclear weapons program. The analyst had the same chatbot write the formal intelligence report. Boarding teams and aircraft were readied before anyone checked the underlying sourcing, and the operation was halted minutes before execution. The claim was false. Nobody in that chain was persuaded by an argument. They were moved by a report that read like a report.
Rank the signals by their distance from the model
Here's the rule I'd build on. A signal about an AI's output is worth exactly as much as its distance from the model that produced it. Tone comes from inside the model, so it's worth zero. Ground truth comes from entirely outside it, so it's worth everything. Everything else sits between those two, and you can rank it.
The bottom is tone. Sure, measured, apologetic: all of it is generated, all of it is style. Strip the adjectives and read what's left.
One step up is the stated number. Ask an AI how confident it is and it'll give you a figure. That figure came out of the same process that produced the answer. It's a prediction of what a confidence number should look like here, not a measurement of anything. There's one asymmetry worth keeping, though. A model that volunteers doubt is telling you something, because doubt isn't the default voice. A model that sounds sure is telling you nothing, because sure is.
Next is the re-run. Ask the same question again, reworded, in a fresh session. Full Fact found answers shifting from day to day, and that drift is useful: two different answers means at least one is wrong, and you just caught it for free. Two matching answers prove less than people think. They mean the answer is stable, not that it's true. Plausible is just as stable.
Above that is the receipt: a source you can open. A citation moves the claim partway outside the model, but only once you've opened it and the source says what the model says it says. Unopened, a citation is tone wearing a footnote. The lawyers sanctioned in New York in 2023 for filing six fabricated cases had citations. They had reporter volumes and page numbers. What they didn't have was anyone who opened them.
At the top is ground truth: the output compared against a record the model didn't write. A database lookup, a calculation you ran yourself, the actual document, the test on the actual system. This is the only reading that tells you whether the output is correct, rather than whether it's consistent or well-dressed. It's also the expensive one, which is why Issues 002 and 019 spent so long on triage. You build this check in where the stakes justify it and skip it where they don't.
What this changes about AI oversight
Two things. First, "the AI seemed confident" stops being a data point in any review. It isn't an input. If someone offers it, ask what reading they took instead. Second, the ranking tells you where to spend. Tone and stated confidence are free and worthless. Re-runs are cheap and catch some errors. Receipts cost a minute and catch many. Ground truth costs real money and catches the rest. Oversight is deciding, per output, how far up that list the stakes push you.
I spent 25 years making automated systems trustworthy in production, and the one rule that never changed was that a system doesn't get to grade its own work. The test comes from outside. An AI that sounds sure is a system grading its own work, in a very convincing voice. Read everything about it except that.