Why Are Voice of Customer (voc) surveys fundamentally flawed?

Why Are Voice of Customer surveys fundamentally flawed?

Voice of Customer (VoC) surveys are a popular, but deeply flawed for organisations to monitor customer preferences and levels of satisfaction. They follow a standard process of using qualitative research to identify issues and then undertake a highly structured quantitative survey to measure importance and satisfaction.

The VoC survey framework appears to be an outdated and false view of people as rational, independent agents with preferences. Because of the way the human brain works people have limited access to the emotional, social and psychological motivations that drive much of our behaviour.

source: iSixSigma.com

Surveys also ignore the power of contextual influences. VoC programmes are highly likely to generate unreliable and misleading results which contribute to an illusion of understanding customers.

A flawed measure:

• Asking direct questions is unreliable. Many of our daily decisions are made without much conscious thought and we post-rationalise decisions to ensure consistency with our internal model of the world. When people answer direct questions they attempt to rationalise their decisions and behaviour. They naturally construct a narrative that explains their actions in a rational and consistent way.
• However, this fails to capture many of the underlying motivations of human behaviour. Furthermore, as our brains use mental short-cuts (heuristics) to save energy and speed up decision making, we are prone to making sub-optimal and often irrational decisions. This is contrary to how we like to perceive ourselves and naturally we don’t articulate this when answering survey questions.

Availability Heuristic

• Asking people about importance is especially misleading. Psychologists have found that relative importance of an item is heavily influenced by the ease with which we can retrieve it from our memory. This often correlates heavily with how easy it is to recall as issues (see availability heuristic) which can be caused by media coverage. I noticed this when I managed a survey of financial advisers. The importance of financial strength always shot up after media coverage of any kind of financial crisis.
• Our ability to recall an event is also limited as a memory is a series of brief fleeting moments from an experience. We construct a memory from visual snapshots, thoughts, feelings, smells and sounds. The perception of an experience is a range of factors including our mood, social context, vocabulary, the physical environment and our knowledge.

Source: Freeimages.com

The Reconstruction Process

• The reconstruction process itself is also the product of these factors. This means that each time we recall an event we have to piece these elements back together. Inevitably our memory changes each time we recall an experience. Indeed, we frequently have false memories of events that we will passionately defend as our brain attempts to retain our internal consistency with our values and beliefs.
• Our memory is heavily influenced by the peak-end rule. We remember what happened during the most intense moment of an experience, and what occurred at the end. The length of the experience appears to have little impact on our overall satisfaction with an event.

Herd Instinct

• Our herd instinct means we are heavily influenced by what other people in our network and beyond are doing. We are constantly copying people, especially when we find ourselves in a new or uncertain situation. Social norms and trends are also powerful forces. We may not be consciously aware of these influences and so we cannot expect people to articulate this via direct questioning.

Source: Freeimages.com

Analysis and Action:

VoC surveys also fail to deliver when it comes to analysis of results. Little or no allowance is the fundamental flaw in using direct questions to obtain the data.

• Due to the survey design operational and customer facing areas often find VoC feedback to be too generic and therefore not actionable. They want to drill down to specific locations or touch points. Sample sizes are not sufficient to allow for such analysis.
• Unfortunately research executives can sometimes be pressurised into providing analysis upon tiny sample sizes. This gets quickly circulates around the organization to support changes in service or product delivery. The fact that responses are unlikely to be an accurate reflection of real customer motivations and preferences is usually ignored until results contradict what a senior stakeholder wants the data to say.

Setting Targets

• Sponsors like to set targets to improve VoC scores but ratings tend to fluctuate for no obvious reason. When there is expensive changes to the product or service is it surprising that we may not see any change in customer satisfaction?
• Some organisations now base changes on an understanding of our memory bias by focussing on the peak and end moments of an interaction. This may lead to improved VoC scores but does this really mean that we are delivering an improved customer experience? Are we not just kidding ourselves by using human psychology to temporarily boost a score?

Experiments and Observation:

Top retailers have known for decades that if you want to find out running a test in a number of stores is more reliable than asking people direct questions. This was the original A/B test and is the reason why online retailers are now some of the biggest users of this experimental research design.

“Just following consumer wishes leads to replaceable products, copycat advertising, and stagnating markets.” – Stephen Brown, Professor at the Kellogg School of Management.

VoC surveys are particularly problematic. The standard framework developed as part of the Six Sigma methodology gives it an aura of validity that the technique does not merit. This is an illusion which Six Sigma followers espouse due a lack of understanding of basic human decision-making and psychology.

Source: iSixSigma.com

More emphasis on conducting experiments, together with observing or listening to real customer interactions would be more effective. Co-creation can also be a powerful approach to allow brands to open up a two way conversation with their customers. Direct questioning for VoC programs needs to be used sparingly and the results treated with extreme caution.