Why Models Hallucinate
Fluency is probability, not truth — the mechanics of confident fabrication, and a taxonomy that tells you which fix fits which failure.
A language model's only native drive is producing a plausible next token. Nowhere in that machinery is a fact-checker: no database lookup before asserting, no internal flag distinguishing 'I know this' from 'this is the shape of a thing people say.' Ask for a citation and the model produces something citation-shaped — authors who plausibly exist, a title with the right cadence, a year in range — because that is what continuations of your request look like. Fluency and truth are correlated in training data, which is why the model is right so often. But the correlation is the entire mechanism. There is no oracle behind it.
Training explains the rest. Facts appearing once in a trillion tokens can't be stored reliably — long-tail questions get long-tail accuracy. Sources conflict, and the model absorbed every side. Knowledge stops at a training cutoff while your questions don't. And the incentives have been perverse: preference tuning and benchmarks historically rewarded a confident guess over an honest abstention, so models learned to answer like students who lose points for blanks. Research published across 2024-2025 made this explicit — under most scoring schemes, bluffing was statistically the winning exam strategy — which is why newer Claude and GPT versions abstain more, and why your prompts should make abstention cheap.
A taxonomy makes the problem tractable, because the fixes differ. Factual fabrication: invented papers, APIs, court cases, people — the model filled a gap with a plausible shape. Faithfulness failure: you provided the source and the summary contradicts it — a grounding problem, not a knowledge problem. Propagated reasoning slips: one early arithmetic error carried forward with full confidence. Context-induced errors: your leading question or false premise ('why is X true?' when X isn't) got politely accepted and elaborated. Same symptom — confident wrong output — four different diseases, and treating fabrication with a fix designed for faithfulness wastes your week.
The strategic consequence: hallucination is reducible, not eliminable — it's the flip side of the generative machinery itself. So engineering for truthfulness is three moves, and the rest of this module is those moves. Grounding: put verifiable sources in context and require answers to cite them. Calibration: give the model permission and vocabulary to say 'I don't know' — the next lesson. Verification: check outputs with critique loops, ensembles, and test sets before they reach users. Teams that ship reliable AI features don't have models that never hallucinate; they have pipelines that catch it when they do.