Why Should You Stop Using Focus Groups?

Why are focus groups problematic?

When I first set out as a client-side Customer Insight Manager focus groups were frequently suggested to me as a suitable exploratory method of research. Initially this wasn’t a problem. I was new to the industry. However, over the years I gradually became more aware of their frailties. I have come across some excellent moderators. But I have also seen for myself how focus groups can be misleading and misused.

Key Problems I Have Experienced:

• Respondents often don’t know what they want. They tend to respond favourably towards what is familiar to them rather than what might be new.
• Respondents sometimes say what they think you want them to say. I have observed the same respondent say completely contradictory things during a group. Seemingly to please the audience.
• Reactions to stimulus material is about how the material is shown and communicated. This can be worse if the moderator isn’t equally comfortable with all the variants.
• Focus group respondents will over-analyse and rationalise topics (e.g. advertisements) when we are looking for their emotional response. This is often the key driver of the desired behaviour.
• A single respondent in focus groups may introduce new information into the discussion and it can completely change the direction of it. Sometimes this can be useful, other times you wonder if it didn’t overly influence other respondents.
• One person can completely dominate the discussion in focus groups. I have observed people changing their opinions to avoid conflict or because they mistake confidence for knowledge.
• Some focus group respondents will not open their mouth once to contribute to the conversation.
• They often take place in a completely alien environment which does not match the context of the discussion. Running focus groups in viewing facilities can exasperate this problem as they can create a laboratory atmosphere.
• Focus group discussion guides often have too many items for discussion and include closed questions. I find moderators are generally overly optimistic about getting through a long discussion guide or don’t suggest removing items to allow for additions to it.

Source: Freeimages.com

More About Focus Groups

The industry could argue that the issues can be dealt with, by an expert moderator. But we now have increasing evidence from behavioural economics that people are often irrational in their decision making. Humans rely heavily on mental short-cuts that allow them to make fast, automatic decisions that require little cognitive effort. Focus groups assume people are rational.

The research indicates that our conscious mind may not be aware of what drives a lot of our decisions. Such findings support the view that people post-rationalise what drives their decision making. This leads to people creating erroneous reasons for decisions when asked direct questions. Neuroscience also supports the view that human decisions are from the unconscious mind.

Indeed, it appears only a small proportion of decisions are made using our slower, more cognitive mind (see System 2). This uses a lot more energy than our unconscious mind. For this reason we use System 2 for complex, difficult tasks that requires a lot of thought. This has major implications for how we conduct and analyse market research. But what else can we learn from behavioural economics?

Key Observations That Have Implications for Focus Groups and Market Research:

• Everything is relative and people don’t know what they want until they see it in context. Priming, anchoring, and framing are strong influences on how we respond to stimulus. People like to compare things that are easily comparable, and the context of how we present items heavily skews how we respond to them. This is especially problematic for anything totally new and often focus groups are employed to test new ideas.
• Social norms such as reciprocation and commitment and consistency are powerful influences on our behaviour. We are more strongly motivated by social rewards than by monetary rewards. Yet most participants in focus groups only want the cash.

Source: Freeimages.com

• Our state of mind affects how we respond to questions. Dan Ariely’s experiments demonstrated how men’s responses to questions are heavily influenced by how aroused they feel. This begs a question for market research. If we ask a respondent questions away from the natural context of the decision does this invalidate their answers? Context is an important aspect of how we respond to stimuli and we may not gather the key insights if we divorce the research from the natural decision-making environment. Focus groups held in viewing facilities lack this contextual background.
• If uncertainty exists people look to the actions of others to guide them. People herd and like to do what other people are doing or saying. ‘Groupthink’ can also set in if the group is too homogeneous and the moderator seeks a consensus of opinion rather than having a show of hands. This is particularly worrying for focus groups.

More Key Observations

• People have an extreme tendency to comply with the commands of someone in authority. If a respondent in a focus group comes across as something of an authority on the subject under discussion you may have a problem.
• Don’t underestimate the importance of presentation. It sets expectations for how people respond now and in the future behaviour. This may have implications for using mock up products or artwork rather than the finished design.
• First impressions count. Once a price is set, in our minds it determines our perception of current and future prices. This indicates how people have a natural tendency to anchor prices from past experiences. Focus groups explore opinions towards early stage concepts using rough illustrations.
• Behavioural economics challenges the validity of relying solely on asking consumers direct questions, and highlights bias that can result from group dynamics (eg herding & groupthink). The most fundamental challenge for market research is that humans are using two different cognitive systems for decision making.

Consumer.ology

Responses obtained from a person using their slow, rational system will be invalid. If in reality people normally use their fast, unconscious brain to process and interact with the stimuli presented. But what if they switch between the two systems at various points?

The consumer’s reliance on the unconscious mind for decision making and also the unreliability of traditional research methods is analysed by Philip Graves in his book Consumerology: The Truth about Consumers and the Psychology of Shopping.

Phil suggests that the most reliable methods for understanding consumer behaviour are live tests and covert observation. Where practical I think this is excellent advice. For websites online experiments can run relatively easily using A/B and multivariate testing. Where live tests are not feasible, there is strong evidence to suggest Predictive Markets, such as those developed by System 1 Research, offer us a reliable alternative.

For a number of reasons we are still going to want to ask consumers direct questions, though not necessarily in focus groups. What behavioural economics suggests to me is that we need a framework in place to increase the likelihood that our methods and analysis will uncover the key insights.

Consider These Factors

1. The mind system that predominately engages during the process.
2. The state of mind and context of the process.
3. How we present items to avoid priming, anchoring and framing bias.
4. Existing behaviour – what habits and preferences currently exist.
5. Emotional responses.
6. Social norms that may be influential.

Under the right conditions direct questioning of consumers can still provide useful insights. A good example is how Proctor and Gamble investigated the potential market for Febreze. Initially Febreze under-performed and was very close to being cancelled. To identify what was holding the product back P&G sent researchers into peoples homes to observe and record cleaning behaviour.

Researchers were able to ask questions during or straight after cleaning episodes. Respondents were therefore in the right frame of mind and in a natural environment to discuss cleaning related products. What they discovered was that people were not aware of their own house odours. P&G’s strategy had been built around removing such unwelcome odours so this is explained why this wasn’t working. Further in-home observation and questioning identified how Febreze customers incorporated the product into existing cleaning habits.

This was a break through as they had been trying to create a new cleaning habit for Febreze. Instead P&G was able to piggy back onto an existing cleaning habit. This resulted in a completely new advertising strategy, and P&G has never looked back. Febreze now has annual sales of over $1 billion.

For focus groups, the dynamics of groups and the environment appears to make their findings more prone to bias. As Philip Graves points out:

“The artificial nature of the research environment can also be responsible for not flagging up something that, in the real purchase environment, is unconsciously reinforced and hugely significant in determining a product’s fortune.” Philip Graves, Consumer.ology

Jonathan Ive, Apple’s Senior Vice President of Industrial Design, once said that Apple had found good reason not to use focus groups because:

“They just ensure that you don’t offend anyone and produce bland inoffensive products.”

Does this mean that focus groups are dead? I suspect not and experiments with using gamification in focus groups have shown how we can use this technique to disrupt and counter dominant group members. What this suggests is that we should look to incorporate some of the insights from behavioural economics into the design of focus groups.

Why not try running a focus group in a kitchen environment if the topic is about cooking appliances? Use role play and gamification techniques to bring out the appropriate state of mind. Observe body language and facial expressions, not just what people say. But most important of all don’t use focus groups as your default method of research. Only use them when they are appropriate and can provide some true insights for your organisation.

Thanks for reading. I hope this blog has challenged a few myths and generated some useful ideas.

Further reading: