illusion of asymmetric insight

Fascinating.

volatility

This post relates to NTTATS, but focused on our experience of life, and in particular our experience of volatility.

I have a core hypothesis that humans appreciate order, that we, in anticipation, expect our experiences to proceed linearly from our past, in an orderly progression, and that in retrospect we "smooth" experiences in our memory to make the past seem more orderly than it appeared in the moment.

Let's talk through some examples.

Consider your sleeping patterns.  How many hours do you sleep at night, taking as your data set the last 3 months?  7 hours?  9?  Chances are you  have a pretty good idea of your average night's sleep.  But dipping into the specific data will vary considerably.  If the average is 8 hours, but some nights you linger an extra half hour awake, playing Bejeweled on your phone, but still wake up at the same time.  That's a 6.25% difference in minutes slept!  And in the moment, when you wake up with 6% less sleep, you may well feel grumpy about it.  But two months later, that incident of less than average sleep will have faded in your memory to seem more or less the standard.

Now consider your moods (or those of someone in your life); looking back over those same 3 months, you may recall being "mostly" happy/content or sad/grumpy, but my suspicion is that your recollection will be seriously colored by your mood at the current moment.  And further, that your expectation for your mood in the near future proceeds primarily from your current state.

The upshot is this (I know this post is wordy, but my mind has been clouded by a sinus infection for days...) - the natural world and our experience of it exhibit significant volatility.  Everything observed closely reveals this volatility, but both society at large and our minds individually seek to flatten out or smooth the data to make it seem less volatile.  Put another way, we try to make the chaotic look orderly.

As with NTTATS, this volatility idea seems really obvious to me after talking it out, but when I first became aware of the concept it seemed novel.  And as with NTTATS, I'm struggling a bit to put into words the practical application of the knowledge.  I think that in both cases the real value is to alter our working expectations to accommodate a reality where the future will necessarily not proceed exactly per past experience.

We need to internalize that volatility is the rule, and not the exception, and then so many frustrating things about the world will look differently.

Let's flesh this out in the comments.  Maybe the pressure in my head will subside soon and I can be more coherent.

Impossible Germany, Unlikely Japan

Wilco's Sky Blue Sky album features a song called "Impossible Germany"; the song begins with the lyric quoted in the title for this post.  It came to mind this morning when I was reading the latest updates on the scenario that continues to unfold in Japan.

The devastation wrought there by the earthquake and tsunami is scary for all of us bearing witness, and the loss of life and pain experienced by the Japanese is heartbreaking to consider.

It also strikes me as a stark reminder of randomness and how we humans are prepared to deal with the random.

My friend AdanA and I exchanged a number of emails last week, some in a connected series and some stand-alone items, and about a variety of topics, but when I look back I see the imprint of the Japan disaster on all of them.  We discussed Japan's adherence to a higher standard in their building codes, and how that decision that assuredly cost more on the front end has ultimately reduced the cost to human life; one has only to look to the recent earthquake tragedies in Haiti and Chile to see what lower standards and lesser quality building materials can lead to (and this is not to be taken as a criticism of the people of Haiti or Chile, but a discussion of randomness and its outcomes.)

AdanA also offered up a question/suggestion, wondering aloud if there may be some mechanism we (people) could employ to compensate for an inbuilt flaw in our statistical understandings...while he did not specifically call out the events in Japan, I felt a connection.  See, even with Japan's relatively robust level of preparedness for both earthquakes and tsunamis, had you asked the average "man on the street" in the days before the 'quake hit what might happen to their nuclear reactor facilities in the event of a "double whammy" scenario, where the area got hit by a 9.0 Richter earthquake and massive tsunami waves, they would likely not have an answer at hand.

I read this morning that the nuclear facility that is in fact failing was built to withstand a quake in the mid 7s on the Richter scale (earthquake magnitudes are measured on a logarithmic scale - a 2.0 is 10 times larger than a 1.0); the quake in Japan clocked in at a 9.0, and the measures in place to keep the reactors cool failed in succession.  Radioactive clouds are escaping from the reactor.  This is a terrible thing.

But is it possible to plan for every contingency?  Is it possible to ever move forward when each step is subject to endless scrutiny and consideration?  How do we train ourselves to be aware of the existence of the highly improbable and to make an explicit accounting of the impact of that highly improbable?  Is it human (I mean in keeping with the way we conceive of our humanity) to say: "Some number of people will die of radiation exposure due to nuclear reactors failing in some number of scenario iterations."?  Is the question of our humanity contingent on the ratio of people dying vs the number of iterations being low enough?

I don't know the answer, but I know these are questions we all need to consider.  Now, for a palate cleanser:

other people's faith

OK, so the article in the New Yorker about Scientology was fascinating (but long, very very long).

Framed as a long form profile of Paul Haggis, a prolific and award winning screenwriter/director, the article uses Paul's longtime association with Scientology to segue into a bit of an expose` of the church.

Haggis had been an adherent for much of his life, and had attained a high level of status in the church (in Scientology, believers progress through a highly organized system classes and grades).  He had also been a very vocal and financial supporter of the church until a family issue brought him into conflict with church and he began a very public and at times vitriolic separation from Scientology.

Not only is the mini-bio of Haggis interesting (in my experience, I generally assume that people who enjoy a high degree of artistic success do not simply burst onto the scene one day and crank out a hit, but it is always revealing to see how long it takes for some people to see success), but the story of his relationship with his chosen religion provides a crash (ha!) course in cognitive bias and the intersection of faith and human nature.

“I had such a lack of curiosity when I was inside,” Haggis said. “It’s stunning to me, because I’m such a curious person.” He said that he had been “somewhere between uninterested in looking and afraid of looking.” 

and

“I was in a cult for thirty-four years. Everyone else could see it. I don’t know why I couldn’t.”

There are more and better quotes in the piece that speak to my ideas, but I'm typing one handed, holding my little sleeping son in the other arm right now, so my internet navigating skills are compromised.  Suffice it to say that I see that there is always an emic/etic consideration with religion - by definition, really - and that consideration, or the tensions that arise from the emic/etic disconnects, have to be addressed in the quest for understanding.

And here's some Bible for you:

Test everything. Hold on to the good.

beautifully rendered video of a bias

A Mystery: Why Can't We Walk Straight? from NPR on Vimeo.

statistics = ignorance?

This post on BoingBoing today provides a good opportunity to explore a pet peeve subject for me: statistics are a problem.

It's a common joke to say "90% of statistics are made up" - common, but funny.  It's also common to claim that "correlation does not imply causation", which seems to be the main message of the BoingBoing post.

My primary beef with statistics is that humans do not seem to possess an innate ability to intuit statistical truth; we almost seem predisposed to "short cut" to conclusions, regardless of what the statistical evidence is trying to tell us.  My secondary beef with statistics is that is almost absurdly easy to game the system; whether you are studying biological systems, economics, sports...in every case you can create a statistical set of data that seems to support almost any contention.

Unfortunately, we exist in this space where one of our best tools for understanding and analysis is a deeply flawed tool.  The example from the BoingBoing post is perhaps a little simplistic, where a skeptical reader of data can apply some "common sense" to sniff out the next level of macro data hiding behind the surface level, but many studies of sufficiently complex systems are challenged by the inability to "step back" and see the subject in a wider context.

So what to do?  We have a flawed tool and a problem with confidence in the outcomes of using that tool...

and as XKCD suggests, sometimes correlation doesn't necessarily mean causation, but rightly does suggest the possibility of a relationship.

The upshot from me is that one needs to take statistics with a grain of salt, and just as in other areas discussed in this blog, one needs to be aware of the bias built in, both in the statistics and in the mind of the audience.

*****************
update: After I posted this I wondered if more examples of faulty statistical reasoning would be helpful...so would they?

and I ran

3.18 miles in a little under 30 minutes and in 30 degree weather...and it felt great.

This should be an unqualified positive, but when you are neurotic like I am, you end up asking some annoying questions:

• IF I enjoy it and it's good for me, why do i go long periods without running?
• what changed in me from when I was NOT A RUNNER to when I suddenly was?
• why are people resistant to the idea of running barefoot or minimalist?
• how does conventional wisdom become conventional?
• how does common sense become common?
• once wisdom is conventional or sense is common, is it possible to lose that status?

(The last three are not necessarily related to running)

The Wife* asked me last night to confirm our decision to circumcise our new baby boy (he's still not here, for those of you keeping track) and I said "Well, we should, um..." and my brain ran into the wall.

See, we had already made this decision, complete with hand wringing and deep thinking, and in the end it more or less came down to "he's going to look like Daddy"  (TMI?).  We both like to think that we are open minded, culturally aware, progressive, and appropriately sensitive, and both of us (one of us more than the other, but still) are resistant to doing things just because "that's what's done".

tell me a story

Thinking more about cognitive bias today and how our brains run in some directions time and again.  Earlier, I touched on the survivor bias and on confirmation bias; today I want to talk about the way people's minds seem to like a good story.  Taleb calls it the "narrative fallacy".  Terry Pratchett makes use of a similar idea in his Discworld novels.  The basic idea here is that people* like for sequences of events to "fit", to "make sense", and to conform to a clean line of causality.  When trying to explain how something happened, we look back at the available data and essentially pick those bits that fit into a story.  Taleb mainly focuses on our attempts to apply a causal trail to explain unexpected "Black Swan"-style events, but it applies more generally as well.  I'm going to suggest that this bias has it's root in humanity's basic need for cosmology.

For example, say you are tasked with writing an essay explaining the causes of the US Civil War (or WW2, or the migration of the Irish in the early 1900s, or anything else).  If you are starting with zero preconceptions about the answer, you might read some other people's explanations (this can lead to clustering, a phenomenon I'll address in a later post), but at some point you will look at the mountain of data available and start picking some. In the Civil War example, it's likely that slavery, economics, taxes, and regional differences in the evolution of industrial capacity will come to play a role in your explanation.  The effect of the bias towards a narrative will influence your essay to come to conclusions that fit together nicely; the run is that reality as experienced in forward progressing real-time rarely follows a clean narrative.  "Some people kept slaves; other people felt that slavery was incompatible with a basic view of human rights; the two parties fought over the issue."  All of the steps in the preceding are true enough in their own right, and in some sort of macro view of one part of the bundle of issues present in the years preceding the US Civil War these data points create a compelling narrative.  The problem is that the story may be the one true explanation of the cause of the Civil War; it may be a true piece of the portfolio of causes; or it may not have figured in the actual sequence of events in any straightforward way at all.  Life, lived in real time and in forward progression, is often more nuanced (and conversely, sometimes much less nuanced!) and more "random" than any narrative we are able to construct afterwards.

Back to Pratchett - several of his books refer to the tendency for people to respond strongly to known stories.  For example, in the book Witches Abroad, the protagonists are fighting against the power of fairy tale stories that are sweeping events along per the fairy tale pattern (princess meets prince, events conspire against their joyful union, prince and princess overcome obstacles and live happily ever after).  The "story" resonates with people and overrides their ability to be critical or skeptical; "everyone knows" that the witch in the story is an evil hag from the woods and will be overcome; "everyone knows" that kissing the frog results in the princess having a handsome prince on hand to marry.

The power of the story and our desire for an apparent narrative also plays into creation of conspiracy theorists.  For some people, when looking back at the events of September 11, 2001, it just seemed to "make sense" that the US government had to have prior knowledge, that our president and Congress must have played some role.  Adding the government's involvement to the events around 9/11 helps to "make sense" of it all, for some people, by filling in some gaps in "the story."

As with other instances of innate bias in our thinking, half the battle is just being aware of the existence of the bias.  Conclusions derived under the influence of a bias are not necessarily wrong or flawed, but have to be evaluated with consideration given to that influence.


____________________________________________________________________________________
*I always hesitate when writing something general like this, expecting that some in the audience will challenge me to explain who "people" are (or "they" or "most of us", etc).  I hope in the space of this blog to keep generalities to a minimum and to only use them when it seems safe to do so.

tell me I'm right

Another quick hit on the cognitive bias topic; let's talk "confirmation bias", because it's everywhere.

But first, this deck of slides was an early (and robust!) source of inspiration and education for me about cognitive biases.  Be warned, it is a wide and deep resource, and can be a major timesuck for the curious.

So, "confirmation bias".  The wife says that its like the opposite of "buyer's remorse" (if you are unfamiliar with buyer's remorse, go find a few of your friends that spent $500+ on an iPad and ask them how they feel about it now).

The thumbnail is that people commonly seek out "confirmation" of a thesis they hold.  The thesis could be that buying something was a good idea (and so the opposite of buyer's remorse, per my wife), or the thesis could be an idea: "Coldplay is the bestest band EVER".  The way this proceeds, for example:

1. person hears "Yellow", a single from Coldplay's first album Parachutes
2. person is overwhelmed by the awesomeness of a ringing D chord
3. person calls their buddy: "have you heard "Yellow"?  It's awesome, you have to hear it"
4. if the buddy agrees, everyone relaxes in a warm glow of "Yellow" together
5. if the buddy doesn't agree, person likely calls a second buddy: "hey, have you heard..."

Confirmation bias afflicts people in every profession and in so many different kinds of situations that once you become aware of the risk of this bias, you will start to see it everywhere.  You may even try to get other people to confirm the presence of this bias...  =)

Don't try to fit the data to the thesis.  If you care about the data, let it say what it says.

Defense wins championships - I know it!  And I can prove it:

just look at the '85 Bears, or the Steelers in the 70s!

What?  The '99 Rams?  The '06 Colts?  Um...exceptions that prove the rule?

the way your thinker leans

I've become increasingly interested in the way that people think.  One of my friends back in college, RD, wrote a capstone paper on the epistemology of thought and it freaked me out.  Since then, I've occasionally sought out, and occasionally been sneaked up on by, reminders that the way people think and the reasons they think that way is complicated and interesting and freaky.

So I hope to explore, over time and in this space, several different kinds of "cognitive bias"; for me talking through things, trying to explain them in plain language, or just describing them again and again but with different words is helpful in my own understanding.

So let's start with "survivor bias".  The basic idea here is that when studying something, anything, you have to pick a data set to work with; choosing your data set from too slim a portion can strongly bias the outcome of the study.

Real world example?  Say a high school is concerned with their success with the AP History test; they want more students to score 4s and 5s (is that still the scale?  sorry of this example is dated!) in order to get the college credits for the class and raise the high school's status relative to its peer group.  So they start a study: when the scores come in for a given year's test, the find that there were 10 students (out of a group of 40 taking the test) that scored a 4 or better - pretty decent results!  So they complete an in-depth of these 10 kids, learning about their diets, study habits, elementary school grades, the marital status of their parents...  Ultimately the school comes up with a lot of data, but they want to be careful before coming to conclusions, so they decide to do the same study the next year, and so get a second set of data from the high scorers (this time, 12 of 39 students scored 4 or better!).  So let's say the school continues this pattern until they have a pretty large sample of students to study, maybe 100+ over 8 or so years.

With all this data in hand, the school runs the numbers and comes up with some insights on "what kinds of students" score 4s and 5s.  What's the problem with the study at this point?  What if the qualities observed in the "winner" group are also present in a significant number of the "losers" as well?  The data set was too narrow and biased to the "survivors" of the process being studied.

Much shorter example: say you want to discover the average time it takes a male aged 22-26 to finish the NY Marathon in 2010.  The race organizers provide you with raw data of finish times and start times for all the guys of that age range.  Simple, right?  But what about those guys that start but never finish?  Pulled hamstring, stopped to chat up a cutie at the water station, decided that running is for cheetahs...whatever the reason, some guys just don't finish the race.  If the data of all starters of the appropriate age range are not considered, there is a "survivor bias" in the results of the study.

At this point, I feel it's important to point out that a study can intend to study only survivor's; so long as the study presents the findings as biased towards the survivors, there's nothing inherently wrong with studying a subset of the available data.  But a persistent problem with statistics and sound bite "findings" of studies is that people tend to misunderstand and/or abuse the findings.

Through careful analysis it has been found that 100% of the survivor's of the Titanic's sinking had urinated at least once during the 24 hours preceding the iceberg incident...if only those poor drowning bastards had thought to go pee!