Thinking more about cognitive bias today and how our brains run in some directions time and again. Earlier, I touched on the survivor bias and on confirmation bias; today I want to talk about the way people's minds seem to like a good story. Taleb calls it the "narrative fallacy". Terry Pratchett makes use of a similar idea in his Discworld novels. The basic idea here is that people* like for sequences of events to "fit", to "make sense", and to conform to a clean line of causality. When trying to explain how something happened, we look back at the available data and essentially pick those bits that fit into a story. Taleb mainly focuses on our attempts to apply a causal trail to explain unexpected "Black Swan"-style events, but it applies more generally as well. I'm going to suggest that this bias has it's root in humanity's basic need for cosmology.
For example, say you are tasked with writing an essay explaining the causes of the US Civil War (or WW2, or the migration of the Irish in the early 1900s, or anything else). If you are starting with zero preconceptions about the answer, you might read some other people's explanations (this can lead to clustering, a phenomenon I'll address in a later post), but at some point you will look at the mountain of data available and start picking some. In the Civil War example, it's likely that slavery, economics, taxes, and regional differences in the evolution of industrial capacity will come to play a role in your explanation. The effect of the bias towards a narrative will influence your essay to come to conclusions that fit together nicely; the run is that reality as experienced in forward progressing real-time rarely follows a clean narrative. "Some people kept slaves; other people felt that slavery was incompatible with a basic view of human rights; the two parties fought over the issue." All of the steps in the preceding are true enough in their own right, and in some sort of macro view of one part of the bundle of issues present in the years preceding the US Civil War these data points create a compelling narrative. The problem is that the story may be the one true explanation of the cause of the Civil War; it may be a true piece of the portfolio of causes; or it may not have figured in the actual sequence of events in any straightforward way at all. Life, lived in real time and in forward progression, is often more nuanced (and conversely, sometimes much less nuanced!) and more "random" than any narrative we are able to construct afterwards.
Back to Pratchett - several of his books refer to the tendency for people to respond strongly to known stories. For example, in the book Witches Abroad, the protagonists are fighting against the power of fairy tale stories that are sweeping events along per the fairy tale pattern (princess meets prince, events conspire against their joyful union, prince and princess overcome obstacles and live happily ever after). The "story" resonates with people and overrides their ability to be critical or skeptical; "everyone knows" that the witch in the story is an evil hag from the woods and will be overcome; "everyone knows" that kissing the frog results in the princess having a handsome prince on hand to marry.
The power of the story and our desire for an apparent narrative also plays into creation of conspiracy theorists. For some people, when looking back at the events of September 11, 2001, it just seemed to "make sense" that the US government had to have prior knowledge, that our president and Congress must have played some role. Adding the government's involvement to the events around 9/11 helps to "make sense" of it all, for some people, by filling in some gaps in "the story."
As with other instances of innate bias in our thinking, half the battle is just being aware of the existence of the bias. Conclusions derived under the influence of a bias are not necessarily wrong or flawed, but have to be evaluated with consideration given to that influence.
*I always hesitate when writing something general like this, expecting that some in the audience will challenge me to explain who "people" are (or "they" or "most of us", etc). I hope in the space of this blog to keep generalities to a minimum and to only use them when it seems safe to do so.