Narrative Fallacy
Narrative Fallacy
Definition
A narrative fallacy refers to the tendency of people to create simple and flawed stories out of a sequence of facts to make sense of the world. We place too much weight on talent, stupidity and intentions rather than luck. People focus on the few times an unusual event occurs rather than the many times the event failed to happen. This stories shape our view of the past and our expectations of the future.
The term was coined by trader-philosopher-statistician Nassim Taleb in the book The Black Swan. He argues that people are constantly fooling themselves by creating oversimplified and incorrect accounts of the past and believing them to be true.
A compelling story can be dangerous because it creates an illusion of inevitability which underestimates the role of luck in most outcomes. However, our minds prefer a simplistic view of the world because inconsistencies slow down our thought processes and the clarity of our feelings.
Evidence:
Narrative fallacies are partly caused by the powerful “what you see is all there is” (WYSIATI) rule that the psychologists Daniel Kahenman identified in his research into System 1 thinking. People can’t help but use the limited information they have access to as if it is fully comprehensive. They create the best story they can from the limited information they have and paradoxically it is easier to create a coherent story when you have little knowledge. This confirms the old adage that “a little knowledge is dangerous.”
“Our comforting conviction that the world makes sense rests on a secure foundation: our almost unlimited ability to ignore our ignorance.” – Daniel Kahneman
Social media’s obsession with the habits of successful people is a good example of how people mistake random behaviours for a causal relationship. However, if you look at Tony Hsieh of Zappos and Jeff Bezos at Amazon you will find two highly successful businessmen who have totally different approaches to corporate culture and how to get the most out of people.
Implications for conversion rate optimisation:
Storytelling is dangerous for optimisers as it can limit our testing and learning to those ideas consistent with our existing mental models. This means we can fail to consider other ideas that don’t align with our narrative fallacy.
Avoid trying to explain the psychological reasons why one variant beat another. We can never be certain why one design works better than another as we don’t have access to implicit motivations and what a user sees and doesn’t see.
Confirmation bias means that we focus on reasons that align with our existing beliefs and so we may jump to conclusions that are flawed.
Causation Bias means we see a cause and effect relationship in data where none exists. This often happens when we see a correlation and assume a causal relationship without having a hypothesis to explain why this would the case.
How to counter the narrative fallacy?
Create clear hypothesis for experiments based upon evidence before proceeding with a test. This helps avoid hypothesising after the results are known (HARKing).
Be disciplined with data collection and the length of experiments. Don’t cherry pick data points and agree how long to run a test beforehand by allowing for the length of the business cycle. Nor pause tests until you have a high level of statistical confidence and a low error rate (usually below 5%).
Don’t communicate changes in conversion rates that don’t reach full statistical confidence. This prevents people creating narratives from unreliable data.
Have a clear view on the action you plan to take following the test, rather than trying to explain why the result occurred.
Related post:
Narrative fallacy – Why are the habits of successful people a myth?
Resources:
Conversion marketing – Glossary of Conversion Marketing.
Over 300 tools reviewed – Digital Marketing Toolbox.
A/B testing software – Which A/B testing tools should you choose?
Types of A/B tests – How to optimise your website’s performance using A/B testing.
