Video mystery shopping is a contentious research methodology. However, rarely does a month go by without hearing about another mis-selling scandal in financial services or some other regulated sector. Given the ramifications of the 2008 financial crisis you would expect more monitoring of firms behaviour. This is especially important in retail financial services where poor advice can be very costly to customers, It may have long term consequences on their lifestyle.
Given reputation and good will is critical in all businesses. It is essential that your customers have confidence that you have the right controls in place to identify when service levels and advice are consistently below expected standards.
MONITORING CUSTOMER INTERACTIONS:
In one of my research roles I came across a company who assured me that they could measure real customer interactions by sitting next to advisers and recording their telephone conversation with customers. I was told that this would not influence their behaviour because the advisers were dealing with business transactions. Sales management were also comfortable with this approach.
This is clearly a fallacy. There is plenty of research that indicates it is human nature to behave differently, as people are more careful to follow standard procedures. They want to portray themselves in a good light with everyone concerned.
A number of years ago I had to establish a programme of research to monitor service standards and identify training needs for an insurance company’s direct sales force. Each sales person was already evaluated on a regular basis during a normal sales visit. The vast majority of sales people passed these assessments without much trouble. The question was whether these observations were a true reflection of their normal behaviour.
EVALUATING FACE-TO-FACE SERVICE & ADVICE:
To ensure an accurate and detailed evaluation of service standards and behaviour, I established a program of regular video mystery shopping surveys. We recruited people who met the basic customer profile and gave them a very simple scenario (customer need) to follow. This allowed the customer to use their own details as much as possible.
THE RESULTS:
Sales management and Legal were shocked by the findings. There was a total lack of consistency in the sales process and most advisers did not perform to expected standards. This was valuable feedback though. A great deal of effort went in to improving the customer experience and the standard of advice given.
IMPLICATIONS FOR RESEARCH:
Only video mystery shopping (using a hidden camera) can accurately capture every element of customer interactions. This ensures there is no disagreement about what was said and done. The sales process in FS is too long and complex to be able to rely on the memory of a mystery shopper.
It is difficult for anyone to ignore video mystery shopping of poor or misleading advice. This is partly because video mystery shopping evidence can capture the whole sales experience. Regulators have access to material when they visit companies and they will expect to see evidence of corrective action.
There was no correlation between customer satisfaction and the adviser’s compliance with basic standards for advice giving. This is perhaps not surprising as why should a customer understand details of the advice process. It also confirms that Voice of the Customer research should not be used as an indicator of the quality of advice provided.
Video mystery shopping can be a powerful aid to change. The videos allow advisers to see the interaction from the customer’s point of view. They encourage changes in behaviour that would otherwise be difficult to achieve. Each adviser was able to view their video with their manager and agree a detailed personal development plan to address any issues identified.
To evaluate the output from video mystery shopping you need the input of trainers to assess the quality of service. Each company has its own standards. You need a good understanding of regulatory requirements to accurately evaluate the service and advice.
OVERALL CONCLUSION:
The findings suggested to me that the regulatory framework is too complex and prescriptive. The focus should be on giving customers good service and advice rather than ticking boxes and following set procedures. Video mystery shopping is the most effective way of demonstrating how complex the process is from the customer and adviser’s perspective.
People need to be able to trust advisers, particularly when dealing with money issues. However, a minority of people will always have a tendency to cheat or cut corners. There also needs to be controls in place to spot such rogue behaviours. It is human nature and it happens in every sector. Video mystery shopping is an excellent tool it can capture the reality of the customer experience and the nature of advice given. With the correct controls in place it can be a catalyst for positive change in an organisation. It is a means for identifying potential mis-selling.
There is a perception that decision making in financial services (FS) is far more rational than with fast moving consumer goods. FS products have more long-term consequences. Decision making is emotional and impulsive. This view is strongly held among FS professionals but is there any evidence to support this perception?
Behavioural Economics & Decision Making:
This view about decision making is sometimes supported by The Consumer Involvement Theory. The theory suggests FS purchases fall into the high involvement and rational segment of the model. This is due to the relatively high cost of FS products and purchases are more about logic and less about emotion. You don’t buy a pension everyday and so decision making must be more methodical.
Source: Freeimages.com
But what does the evidence from experiments in behavioural economics and neuroscience indicate about rational decision making in the face of risk and uncertainty? Are consumers’ really discreet, self-determining individuals who make considered, rational decisions?
This view increasingly looks misguided and is probably a fallacy created by our own minds to make us feel in control of our behaviour. As Mark Earls points out in his book Herd:
“Our failure to acknowledge the truth about human nature distorts our attempts to understand human behaviour and frustrates our attempts to change it. Bad theory = Bad Plan = Ineffective action.” Mark Earls on Stephen Pinker, Herd
Behavioural economists Dan Airely and Nobel laureate Daniel Kahneman have uncovered strong evidence that rational decision making is often an illusion. That is not say people don’t behave differently when considering money issues. Dan Ariely found that just thinking about money makes people more selfish, self-reliant and less charitable. However, these traits don’t necessarily make people more rational in their FS decision making.
Source: Freeimages.com
Insight identified from behavioural economists challenge many of the basic assumptions of traditional economics and related theories of decision making. Some of the Key insights are:
Emotions
Human decision making is unconsciously driven by our emotions and social norms much more than we have appreciated in the past. This is due in part by our reliance on our fast, intuitive, but largely unconscious mind. Daniel Kahneman refers to this as system 1. This makes the majority of our decisions. But its frequent use of rules of thumb (heuristics) make people prone to biases that can lead to sub-optimal decisions.
Source: Freeimages.com
Answering An Easier Question
Because we find cognitive thought hard work, system 1 will often substitute an easier question for a difficult question to answer instead. It will do this automatically if we are unable to easily retrieve an answer to a hard question. This could undermine rational decision making if we find a question or calculation too difficult to consider.
Context Dependency
Our state of mind and decision making is heavily influenced by the environment within which we find our selves. This leads to inconsistencies in our decision making that we are largely unaware of.
Source: FreeImages.com
Memory
Our recall of events is unreliable and heavily biased towards the beginning, the peak of activity and the end of an event. We neglect the duration of an event and have little awareness of our true motivations. Indeed, every time we try to retrieve a memory our brain has to reconstruct it and inevitably this changes what it contains. This explains why sometimes we create false memories that we genuinely believe are accurate.
Illusion of Understanding
Kahneman uses the acronym WYSIATI (What You See Is All There Is). This describes our tendency to think that the limited information we have about the world is all that there is to know. Humans create narrative fallacies in an attempt to make sense of what are often random events. If we don’t acknowledge our ignorance of important information this will again influence the effectiveness of our decision making.
“Our comforting conviction that the world makes sense rests on a secure foundation: our almost unlimited ability to ignore our ignorance.” Daniel Kahneman, Thinking, fast and slow
Source: FreeImages.com
People Herd
As Mark Earls points out humans are a “super social species”. Our behaviour is unconsciously influenced by what other people do (see Herd Instinct) and more so than we realise or like to admit. In the face of uncertainty we look to how other people behave and will often follow their lead. Following the herd can in some circumstances be a rational decision making strategy, but as with stock market bubbles it can also be a recipe for disaster.
Demand Is Social
Mark Earls argues that market size and market share are primarily a function of consumer-to-consumer interaction. The implication being that rather than focusing on supply side factors, marketing should pay more attention to understanding and modelling interactions that generate mass behaviour (e,g, consumer-to-consumer interactions).
“You have to understand the rules of interaction. The accepted behaviours and rules of thumb of the individuals whose interaction generates the complexity of behaviour that you are studying. These will shape the outcome of interactions.“ Mark Earls, Herd
People Care About Others
Real people are also sometimes generous and willing to contribute to the good of the community. These are not characteristics of rational decision making as described by traditional economic theory.
We Think Of a Reason After The Event
So peoples’ decisions are mainly influenced by factors that they are not consciously aware of. Humans review and post-rationalise decisions (see Choice Supportive Bias). This suggests that our perceptions of a product or brand are likely to change after an action. Rather than before as implied by traditional marketing models like AIDA (Attention, Interest, Desire, Action). It should probably change to Context, Attention, Emotion/social norms, Action, Review, Memory (C.A.E.A.R.M). Not a great acronym. I still find marketing people using the old AIDA model so we do need to encourage them to move on from it.
PROSPECT THEORY:
So what specifically does behavioural economics have to say about FS decision making? Risk and uncertainty is at the heart of Daniel Kahneman and Amos Tversky’s Prospect theory. Three cognitive principles form the basis of the theory:
The perceived value of a decision outcome (the utility derived) is dependent upon the history of one’s wealth (the reference point). This may seem obvious,. Traditional economics does not recognise that a poor person will perceive a gain of £1,000 as generating more utility than would a millionaire. A person’s reference point is often the current status quo.
People experience diminishing sensitivity to both sensory changes (e.g. light) and to changes in wealth. So for example the subjective difference between £1,000 and £1,100 is much smaller than between £100 and £200.
Humans are loss averse. When compared against each other people dislike losing more than they like winning. Thus losses loom larger than gains even though the value in monetary terms may be identical. This explains why investors find it painful to sell shares that are below their purchase price. They find it easier to sell shares that are in profit. This is not rational decision making behaviour.
LOSS AVERSION:
Loss aversion is key to understanding how people perceive financial services, and gambling of course. Extensive research estimates the psychological value of losses and gains. These studies have identified a loss aversion ratio of between 1.5 and 2.5. This means that a loss that is identical in money terms to a gain is up to 2.5 times more than the gain.
Source: FreeImages.com
Interestingly, professional risk takers such as fund managers and full-time gamblers are more tolerant of losses. This may be because they are less emotionally aroused than the amateur investor. Loss aversion leads to predictable behaviours.
The Situations:
If a potential loss could be ruinous or would threaten their lifestyle, people will normally dismiss the option completely. Only obsessive gamblers would normally consider this type of situation.
Where people are presented with a situation where both a gain and a loss are possible people tend to make extreme risk averse choices. For example a person is given the choice between a small guaranteed gain over 5 years (e.g. a deposit based account) and a stock market linked product that carries a low risk of a large loss. People have a tendency to focus on the large potential loss and often select the former, less risky option. This is why advisers will focus on the large upside potential of a stock market linked investment. They try to play down any potential for large losses.
Where the choice is between a certain loss and a larger loss that is just a probability (i.e. there is a chance of no loss), diminishing sensitivity can result in excessive risk taking. This explains why private investors sometimes refuse to cut their losses on poorly performing shares. Instead they invest more money (to reduce the average purchase price) in the hope that the price will recover sufficiently to avoid a loss. The sunk-cost fallacy results in a lot of irrational decision making.
“Loss aversion is a powerful conservative force that favors minimal changes from the status quo in the lives of both institutions and individuals.” Daniel Kahneman, Thinking, fast and slow.
THE POSSIBILITY AND CERTAINTY EFFECTS:
When considering FS decision making it also necessary to understand how consumer evaluate risks. There are two key biases that relate to the psychological value (weight) given by people to different probabilities or risks.
The possibility effect results in highly unlikely (low probability) events being given more weight than they justify. This helps explain the attractiveness of both gambling and insurance policies that cover unlikely events (e.g. extended warranties).
The certainty effect leads to events that are almost certain being given less weight than their probability justifies.
Indeed, research shows that unlikely events (1% to 2% probability) are over weighted by a factor of 4. However, for an almost certain event the difference is even larger. In experiments a 2% chance of not winning was given a weighting of 13% (or an 87.1% chance of winning).
RARE EVENTS:
Where the odds of an event are very small (e.g. around 0.001% or less) people become almost completely indifferent to variations in levels of risk. Rather emotional factors and how a risk is framed are the key drivers of how people react to these levels of risk. This explains why after a terrorist attack there tends to be more focus on whether insurances cover such risks even though the level of risk (to an individual) remains extremely low. It also helps to explain why people are often too willing to bet on extreme events happening.
“When the top prize is very large, ticket buyers appear indifferent to the fact that their chance of winning is minuscule.” Daniel Khaneman, Thinking, fast and slow
Kahneman also found evidence that rich and vivid descriptions of an outcome (e.g. the lifestyle of a lottery winner) helps to reduce the impact of probabilities. In particular he found that people are more heavily influenced (in terms of weighting of probabilities) if an event uses frequencies (e.g. the number of people) rather than abstract concepts such as chance or risk.
RISK AND NARROW FRAMING:
Due to our use of intuitive thinking (system 1) and the laziness of system 2, most people have a tendency to evaluate individual risks separately and independently. People tend to make decisions when a problem arises rather than trying to look at the bigger picture. Kahneman suggests that narrow framing is one of the most common causes of poor decision making.
What Kahneman found was that this approach will almost always lead to sub-optimal decisions due to our focus on loss aversion. The best solution is to aggregate decisions together. A professional investor achieves this by always looking at individual shares as part of a balanced portfolio. This reduces the impact of loss aversion on our preferences.
MENTAL ACCOUNTS:
People hold their money in different accounts, some of which are real and some are only mental (e.g. money from my dad to buy my daughter a present). There is normally the everyday spending account, general savings, savings assigned for emergencies, maybe savings designated for private education and so on. People use mental accounting as an aid to self control. They have a clear hierarchy of willingness to use these accounts to cover their immediate needs and have an emotional attachment to the state of their mental accounts.
Mental accounting is a form of narrow framing and can have disastrous consequences in financial decision making. It often leads to private investors to set up a separate mental account for each share they own. This results in investors wanting to close each account as a gain. So when they need money for their daughter’s wedding what do they do? They have a very strong preference to sell winners rather than losers. It also helps to explain why consumers might have an outstanding credit card balance of £2,000 (with an APR of around 20%). Yet have savings of £10,000 (paying just 4% interest). These are not rational decision making behaviours.
Source: FreeImages.com
REGRET:
Emotions are also an important factor in how we evaluate gains and losses. Most theories of decision making assume that people evaluate available options in a choice separately and independently. This does not reflect human nature. People feel regret when the experience of an outcome is affected by an alternative option that was open to them, but they did not choose. Thus missing out on selecting the top performing managed fund may influence the perception of your investment choice.
ARE PEOPLE REALLY MORE RATIONAL WHEN BUYING FS PRODUCTS?
The evidence clearly suggests no. People are prone to the same biases with decision making for FS products as they are when buying consumer goods. Even the result of the 2016 UK referendum on membership of the EU appeared to be more driven by gut instinct and emotion than rational decision making. Similarly, FS decisions are often subject to powerful disruptive forces (e.g. loss aversion and mental accounting) than every day purchases. This demonstrates the importance of regulation to protect people (e.g. cooling off periods) and consumer education in the FS sector.
WHY DO FS MANAGEMENT BELIEVE IN RATIONAL CONSUMERS?
Source: FreeImages.com
Senior management in the FS sector is a series of very numerate professions who estimate risks and calculate probabilities. There are actuaries in life & pensions, underwriters in general insurance and lending, bankers, accountants, and a smattering of economists. Given their training and experience of dealing with risk and uncertainty they are less prone to key cognitive biases such as risk aversion and mental accounting. For this reason FS management are less likely to appreciate how strongly consumer behaviour is influenced by these biases and decision making shortcuts.
I observed an example of this when I worked for a large UK life assurance company. We developed a Guaranteed Capital Bond that protected your initial investment and provided some limited potential to benefit from any rise in the stock market. It researched well, but the CEO (who was an actuary) thought it wouldn’t sell. It didn’t offer enough upside potential if the stock market grew strongly. The Director of Sales & Marketing (a sales person) was supportive of the launch because he understood how loss averse people can be. I don’t need to say who won the argument when it went on sale.
IMPLICATIONS FOR MARKETING:
I could write a whole post on the implication for market research arising from the above insights. Instead I would like to finish with just a few suggestions for consideration:
Use Analytics To Better Understand Current Customer Behaviour
In the digital age we can now use web analytics to track and measure online customer behaviour. We also have the ability to conduct online experiments (i.e. A/B and Multivariate testing). But even in the off-line world there are many sources of data to explore and analyse before we need to conduct primary research.
Fewer Focus Groups Please!
In some FS organisations focus groups appear to be the default research tool. Companies Should stop using focus groups because they tap into System 2 thinking. They have many faults that can lead to misleading findings. Interestingly John Kearon of System 1 Research (previously BrainJuicer) made a similar observation:
“Yes, they (Focus groups) can reveal powerful insights in the hands of a great researcher. But all too often they are just the lazy default of unquestioning research buyers and produce little or no insight on the subject at hand.” John Kearon, BrainJuicer
Don’t Ask Direct Questions – Observe Behaviour
People are unreliable in their recall of why they make decisions because we don’t have full access to the part of our brain than makes most decisions. Insights are more likely to emerge from observing human behaviour during key experiences than trying to ask direct questions. Observational methods such as ethnography and auto-ethnography are preferable. But implicit methods of research such as the implicit association test and eye tracking are now much more cost effective.
Covert Monitoring of Behaviour
There is plenty of evidence to show that people behave differently when they know they are being observed. I used video mystery customers (using a hidden camera) to evaluate training and development needs for one company’s sales team. Almost all of them met agreed standards when they were accompanied on visits by a trainer. However, almost the opposite was observed when analysing the videos of the mystery customer appointments. Unless you have regular monitoring of service standards in place you can’t be sure what level of service your customers are receiving.
Customer Facing Staff
Listening to sales people, advisers, brokers, telephone agents, people who speak with customers on a daily basis can very insightful. People are better at observing how other people behave than trying to explain their own behaviour. Experienced sales people collect a wealth of knowledge about how customers respond to different strategies. Their turn offs, what excites them, what confuses them and what motivates them.
Co-create
A collaborative approach to research encourages mutual respect and shared learning. Including social influencers (i.e people who shape attitudes and behaviours of their peers) in the process helps ensure the generation of more innovative ideas than having only experts and working parties involved. Collaboration also helps break down barriers between different stakeholders and speed up concept development and refinement.
Crowd Sourcing
There is growing evidence that asking large groups of people to participate in predictive markets can be a good way of selecting winners. James Surowiecki’s book, The Wisdom of Crowds, has a mass of evidence to support this approach.
Understand The ‘how-mechanic’ Of Groups Of Consumers
“By examining the interactions and behaviours that a particular group of people has. It is possible to identify the underlying rules that drive it.” Mark Earls, Herd
The mistake many organisations make is to see Word of Mouth (WoM) as a channel rather than the way consumers interact and influence each other. To benefit from this insight it is necessary to understand the conditions of interactions (e.g. the environment) and the rules of interaction (e.g. how people engage with each other). By making small changes to either or both of these elements of interaction we may be able to significantly influence individual and ultimately group (e.g. private investors) behaviour.
Thinking, fast and slow by Daniel Kahneman, Herd by Mark Earls (@Herdmeister), Influence by Robert B. Cialdini, PHD (@RobertCialdini) ; Predictably Irrational by Dan Ariely (@danariely); the Upside of irrationality by Dan Ariely; The Wisdom of Crowds by James Surowiecki; Consumer.ology by Philip Graves (@philipgraves); Nudge by Richard Thaler (@R_Thaler).