Should You Optimise For Your VIP Customers?

What if most revenues are generated by a few VIP customers?

Some websites get most of their revenues from a relatively small proportion of high value VIP customers. This begs the question should you optimise your site design around your most profitable segment of customers or for your mass of ordinary ones?

How do we optimise the conversion rate?

One of the most scientific methods we use to improve site design and increase the conversion rate is through online experiments (i.e. A/B tests and multivariate tests). However, when we run the analysis for such tests the standard practice is to remove outliers to avoid the results from being overly skewed by abnormal observations. Like high value players (VIPs). Is this practice consistent with a website where a small minority of customers generate the vast majority of revenues?

I was recently asked this question on behalf of an online gambling site. 5% of their users generate over 50% of revenues. Here is what they asked:

“How can you reliably test revenue uplifts in an industry which is driven by outliers? We are removing the top 5% of outliers from tests but that 5% of users is generating ~50% of the revenue. So variants could be winning which aren’t suitable for VIPs, and if they don’t like the changes we could lose a lot of revenue!”

Pareto Principle:

As the Pareto Principle tells us most sectors have a similar issue – around 80% of the profit often comes from 20% of customers in many sectors. Online gambling may or may not be more concentrated than this, but it is not an uncommon problem. However, trying to predict who are the high value customers when they first land on your site is more problematic.

Moving Target:

Indeed, a key characteristic of high value VIP customers is that most begin their journey looking and behaving the same as the majority of new visitors. However, survivorship bias means that we have a tendency to ignore this fact and so we concentrate on the characteristics of those who remain rather considering the nature of those who have been eliminated along the way.

For example, a majority of first time deposits from customers who become VIPs are relatively low. The most frequent amount is often on or near the minimum deposit level. Sure, you get a tiny minority who come in with large first deposits, but they are probably already VIPs on other sites or have a windfall. They do not represent the majority of VIPs.

Think about it, if a large supermarket noticed that high value customers shop more regularly and have more items in their basket, would they re-design the store and remove lines only purchased by lower value customers? Nope, that would be stupid as lower value customers might one day become a high value customer. It would also potentially annoy low value customers and and they might shop elsewhere. Higher value customers have the same basic needs, they just happen to have a higher disposable income or a windfall.

High value (VIP) visitors do not represent a fixed pool of customers. It is in a constant state of flux as user circumstances and behaviour change over time. Very few people, if any, will remain true VIP users throughout their customer life cycle. Their income, luck, assets, lifestyle, attitudes and other factors change as people progress through different life stages.

VIP Customer Intent Is High:

Do drug addicts worry about the user experience? Nope, their intent is so strong they will do almost anything to get a fix. Most VIP customers on gambling sites (or other kinds of sites for that matter) are demonstrating similar addictive behaviour.

Like any addict they will jump through hoops to achieve their goal. I doubt very much that many VIPs will be put off by a long form or poorly designed check-out. If they are then god help your other customers.

Conclusion:

VIP or high-value customers certainly need your attention. But that should be through CRM and personalisation to improve their customer experience and retention. However, as such customers are not a fixed group of people you should definitely remove outliers from A/B and multivariate tests.

It would also be counter-productive to optimise a site just for your high value customers. You would potentially turn-off non VIP customers and you would not have the opportunity to nurture customers as they progress through different value segments. In gambling the pool of VIP customers is usually too small to conduct robust experiments and so you would also be in danger of drawing false conclusions due to the law of small numbers.