CogniqOS
Subscribe

CONCEPT #027 ·VOL I The Individual · Judgment & Evaluation ErrorsLEXICON ENTRY

Illusion of Validity

Discovered by Daniel Kahneman and Amos Tversky, 1973 · Origin ↓

The Coherent Story Feels Like Evidence The more consistent a pattern looks, the more confident we feel - even when the pattern has zero predictive power. *(Evidence: Strong)* A cognitive bias where the internal coherence of a narrative or dataset creates subjective confidence that is unrelated to the actual validity of the judgment. Identified by Kahneman and Tversky: people mistake "good fit" for "good prediction." A compelling story with clean data feels reliable even when the underlying signal is noise.
Category: BiasEvidence: ReplicatedTier 1Type: Mechanismⓘ what these mean

This is a Lexicon entry: the mechanism, when it shows up, and the countermeasure. The full correction, with real-world cases and the audit prompt, hasn't been written yet.

When it shows up

Hiring decisions based on structured interview narratives. Investment theses built on internally consistent but cherry-picked data. Medical diagnoses where symptoms form a neat pattern. Sports drafts where highlight reels create false certainty.

Failure mode

Confidence tracks coherence rather than evidence quality. Decision-makers reject superior statistical models because the model's output "doesn't feel right" compared to their intuitive narrative. Interview confidence bears near-zero correlation with actual job performance.

Countermeasure

Separate the question "Does this story make sense?" from "Does this story predict outcomes?" Demand out-of-sample validation. Use base rates and actuarial methods alongside narrative judgment.

Get the pre-decision checklist →

Related concepts

Also connected in the mapThe Narrative Fallacy·Confirmation Bias·The Overconfidence Effect·Subjective Validation·Gibson’s Law·False Precision

Origin

Daniel KahnemanDiscovered by, 1973Daniel KahnemanPsychology · Nobel laureate · 1934–2024Proved judgement runs on fast, biased heuristics - the foundation the whole field stands on.See in Origins →Amos TverskyDiscovered byAmos TverskyCognitive psychology · 1937–1996Kahneman's co-author; formalised how people misjudge probability and risk.See in Origins →

Read next

Want the full correction for Illusion of Validity?

This entry has the mechanism and countermeasure. The full write-up - real cases, sources, the audit prompt - comes later.

Entry #27 of 631 in The Lexicon · see the full Lexicon