Would NPS be better if its scale only had three numbers?
Successful innovation doesn’t have to involve a massive breakthrough in technology – Interview with Adrian Collins of bac< and Ziggurat Brands
September 5, 2014Business needs to be agile and customer centric if it is to avoid the coming customerpocalypse – Interview with Alan Trefler, CEO of Pegasystems
September 12, 2014A question I have been asking myself recently is: Would Net Promoter Score (NPS) be better if it’s scale only had three numbers?
Now, before I get accused of heresy, let me say up front that I am a fan of the Net Promoter Score (NPS) system’s central question:
“How likely are you to recommend [our product or service] to your colleagues, friends or family?”
In fact, I think it is a useful and valuable question the answer to which can help firms better understand the levels of loyalty and advocacy that exists within their customer base.
When asking the question, the NPS system asks customers to rate their likelihood to recommend a product or a service on a zero to 10 scale. Those customers that score the firm zero to 6 (not likely to recommend to neutral) are labelled as Detractors. Those scoring 7 and 8 are labelled as being Passive, whilst those scoring the firm 9 and 10 are labelled Promoters and extremely likely to recommend.
However, on the back of a recent interview I conducted with Nicola Millard of BT, I found myself questioning the value of a zero to 10 scale for a couple of reasons:
1. BT’s Simple Approach – BT have developed a new scoring system, ‘Net Easy’ – inspired by NPS and Customer Effort Score (CES) – which helps them understand how easy it is for their customers to do business with them. Their experience and other industry research shows that if you make it easy for your customers then they are more likely to return and be loyal. More interesting is how their ‘Net Easy’ score system has been stripped down to a minimal three point scale, where -1 is for difficult, 0 is for neutral and +1 is for easy.
2. The Questionable Value Of Some Scores – Michael Lowenstein of Beyond Philosophy made a comment on my interview with Nicola and in it he said that, in studies that they have conducted, only an NPS score of 10 showed a moderate correlation with actual positive and resultant customer behaviour (repeat custom and advocacy). He went on to add that scores of 8 or 9 show ‘virtually no connection to perceived value and future action’. So, if research shows that a score of 8 or 9 gives no real indication of what a customer is going to do in future then, surely, the question must be: what is the point of having a score of 8 or 9?
The problems don’t stop there if we add in the subjective element at play within larger scales. For example, a score of eight, nine or ten, depending on the product or service, will mean different things to different people of different gender, language, age, culture and experience. Personally, I know that I struggle with some of these scales and gauging the difference between being fairly likely, quite likely, very likely or extremely likely.
Surely the thing that is of real value to a customer and a business is whether or not the customer is satisfied, found it easy to do business with them or whether they are going to recommend them. Or not.
So, if we want clearer answers and insights then shouldn’t we be brave enough to ask fewer and more definite questions? Would it not, therefore, be more useful to just ask a customer:
- Are you satisfied: Yes? No? Neutral/Don’t Know?
- Did we make it easy for you? Yes? No? Neutral/Don’t Know?
- Will you recommend us to your your colleagues, friends or family? Yes? No? Neutral/Don’t Know?
Wouldn’t these type of questions and scales not produce more direct, useful and actionable feedback?
Therefore, would NPS (or any customer facing scale for that matter) be better if it only had three numbers representing yes, no or neutral/don’t know?
This post was originally published on my Forbes.com column here.
15 Comments
Adrian,
You are bang on. Michael’s findings do not surprise one piece. Personally I think NPS is misused for insight into transactional quality. Who actually links a moment of truth to advocacy. Maybe when it was a big deal but not in relation to frequency with which it is offered. It is a strategic and occasional question that asks how the relationship over time is doing and therefore if you feel motivated to share that with others.
The transactional stuff is simple. Much more to do with net easy. Did we deliver in all respects – the what and how. If not where did we screw up?
But you main point is bang on. Our minds work best with a yes/no option. Brands are activated better with a yes now option. Do use a score to finesse and try to bump up scores. Get the reality and then ask for specific feedback. That the way to make it work.
Martin
Cheers, Martin. I agree that a better question post transaction would be ….how did we do or how could we do it better. Asking about a willingness to recommend could feel like the ‘relationship’ is being rushed and the question is not relevant.
Adrian
Interesting point Adrian. It’s worthwhile to note that Enterprise Rental Car, the inspiration for NPS, has a five point scale. Enterprise found that the key to future growth were the number of respondents that were “completely satisfied 5’s” because they are the ones most likely to recommend. This aligns with your thinking.
For nine other takeaways / key learnings from Enterprise and their ESQi (Enteprise Service Quality index), here’s a post: http://www.9inchmarketing.com/2013/12/15/key-lessons-in-driving-customer-loyalty/
One thing I’d recommend whether its three point scale or a five point scale or CES’ seven point scale is to add one additional number and that’s zero. This does two things:
1. Eliminates any confusion that #1 may be the highest ranking.
2. Eliminates the wishy washy middle option by establishing a four or six point scale.
Thanks for posing the question. I think its extremely useful.
Thanks Stan for sharing your insight and the link.
My thinking was that if I suffer with extended scales then I can’t be the only one. I mean what is the difference between and 8 and a 9?
However, using a longer scale is easier than using a shorter one as it allows us to cover more eventualities and prevents us from making choices.
I believe there is real value and insight in doing the work and making things shorter and simpler both for business and their customers whatever the scale and the application.
Adrian
Back in 2008, Dr Alain Samson at the LSE and I wrote a paper for Admap titled Customer advocacy metrics: the NPS theory in practice. We interviewed 50 brands along with investigating whether the NPS tells you anything about actual customer behaviour and it’s useful as a predictor of growth – particularly given the controversy surrounding the Marketing Science Institute having awarded Tim Keiningham of Ipsos loyalty and academic colleagues for an article demolishing Reichheld’s arguments. Dividing the scale into 5 or 3 doesn’t really seem to address the criticism about the method’s statistical inferiority to other metrics despite the practical benefits of the approach (short survey, simple concept to communicate). A better suggestion would have been to ask a simple more qualitative question in addition, i.e. why do you score that way. This would arguably tell brands more than any attempt to use metrics to quantify advocacy, and use this as means for predicting growth, customer behaviour, etc.
Hi Justin,
Thanks for weighing in and for sharing your insight. I like your suggestion that a more useful follow up question would be to understand why people scored in a particular way as that helps us understand motivation and issues experienced. However, we still have to decide what question we are asking up front.
I like Martin’s idea of asking appropriate questions depending on the context i.e. how did we do? is a more appropriate question to ask post-transaction.
Adrian
NPS is good, but for me the number is irrelevant. Yes it’s a good indicator whether customers will recommend you or not but all to often the focus is on the number and nothing else.
Of more importance is why people will/will not recommend you – you need to ask this as well. This will provide you with the real insight as to what you are good at and what you may need to focus on or fix.
Without this insight NPS is just another faceless metric with no real meaning.
Hi Jase,
Thanks for getting stuck into the conversation. I agree that understanding what is driving a persons response to a survey is a missing key here. However, this idea doesn’t just apply to NPS but and could applied to any type of survey, don’t you agree? Otherwise we just have numbers and data and not a lot of understanding.
Adrian
Couldn’t agree more Adrian!
Listening (& understanding) is critical to service improvement.
Numbers are good to track to see if any CX initiatives have worked (or not worked!), but that’s about as far as the usefulness stretches.
Good to see you seem have hit a chord with this one though!
Adrian the problem with NPS and other satisfaction metrics is that they attempt to quantity experiences, which are inherently qualitative. That’s useful for managers justifying their bonuses, and consultants justifying renumeration, but clearly questions about why the score was given are going to provide more insight. This can also be achieved by monitoring customer conversations, and there’s now doubt that’s useful for incremental improvement. But where I think I differ with Martin is whether this leads to any breakthrough or real innovations because of faster horses and all that. I think brands are going to take a leaf out UX practice and go beyond the desk and dashboard to find real insight if they are going recreate more agile and responsive customer service. Without truly understand the motivations and needs of their customers, that are not necessarily rational, what are the assumptions they use for analysing any data based on?
Justin, that’s really interesting and can see how brands could benefit from that. It’ll be interesting to see if brands do start to, as you say, take a leaf out of UX practice. I’ll be watching with interest.
Adrian
Thanks, Adrian, for this thought-provoking post. The topic of scales is one of those religious discussions… put five people in a room and get five different opinions or suggestions on what the best scale is. I enjoyed reading everyone’s comments on this.
Annette 🙂
Thanks, Annette.
It felt like a question that needed to be asked.
Adrian
Adrian,
There is an old saying ~ weighing the pig won’t make it fat.
James
James,
Maybe it has something to do with Monday morning but that saying had me stumped so I had to look it up to get a grip on your meaning.
Good point. Thank you.
Adrian