It has been almost four months now since I first blogged about current Net Promoter Score (NPS) debate among market research practitioners and academics.
This month, the debate about NPS continues, as evident in a new critique of NPS’ predictive utility published in the Journal of Marketing. Since its publication, I have found myself involved in a number of interesting email exchanges, particularly with Tim Keiningham (co-author of the article and Senior VP at Ipsos Loyalty) and Justin Kirby (co-author of ‘Connected Marketing’). I have also recently given a podcast interview to Justin Kirby, which covers some of my thoughts on NPS, among other things.
In any event, I very much welcome the dialogue about NPS, because it should ultimately help our understanding of the NPS and other metrics. Research (as done by Keiningham and colleagues) should advance the insights necessary to identify the strengths and weaknesses of different measures. Indeed, as a DIY market research tool, the NPS invites you, the practitioner or academic, to ‘see for yourself’ what it can do for you. As such, (and as anything available in the public domain), it invites you to not only use it blindly, but also test, challenge or even improve it, if you are so inclined!
The main finding by the Journal of Marketing article is that NPS, for the industries covered, does not do a better job at predicting company growth than the NCSB (Norwegian Customer Satisfaction Barometer) or ACSI (American Customer Satisfaction Index).
As a result of this, let me present you with the following:
The Net Promoter Score One-Question – One-Answer FAQ:
Q: Is the NPS the ’single most reliable’ predictor of a company’s ability to grow?
A: Unlikely(!), but it may be a pretty ‘reliable single’ indicator (if one indicator means ‘one question’ here) of loyalty and advocacy, as well as a predictor of growth. If it were the single most reliable predictor of growth, the people behind NPS would, in my opinion, deserve a Nobel Prize in Economics! (Whether false claims have been made by Fred Reichheld, Bain & Company or Satmetrix about what NPS can do is a question that I would like to leave up to your own judgment, provided that you’ve read the NPS critiques and the original HBR article or ‘The Ultimate Question’.)
The ‘Advocacy Drives Growth’ project I’ve been involved in (often cited as ‘proof’ that NPS works in relation to business growth) has shown that NPS can be a statistically significant correlate of company performance. But it did not do a better job at that compared to our measure of negative word-of-mouth, another one-question indicator — at least for the companies and industries we surveyed in the UK.
This makes as much intuitive sense as the NPS itself, because, as I think most of us would agree (and as I mentioned in a previous blog),
1) the more we know (the more questions we ask), the better we usually are at predicting an outcome, and
2) knowing how people actually behave (e.g. re-purchase, recommend, bad-mouth) should do a better predictive job than knowing how people intend to behave.
There are some other important aspects relating to the universal vs particular utility of NPS that have to be (or are already in the process of being) investigated by researchers. This includes a better understanding of those taken-for-granted cut-off points for NPS promoters, passives and detractors, particularly problematic in cross-cultural NPS use.
Be that as it may, in my view (I am in part echoing others, such as Paul Marsden) the NPS is:
- a simple one-question attitudinal indicator of loyalty and advocacy that can be linked to growth
- a ‘reporting tool’ that ‘speaks the language of the boardroom’ and stands for (but does not explain!) one aspect of business health
- a score that can easily be communicated to your employees and customers and thereby increase accountability
- a simple way of benchmarking (assuming you have an unbiased and large enough sample to minimize your measurement error) against the competition, as well as segmenting your customers
A proper NPS audit, especially when it is conducted for the first time, should include one or two open-ended questions asking customers to explain their rating and suggest improvements to the product or service.
I agree with critics who say that the NPS is somewhat of a ‘black box’ and that some businesses have adopted it without truly knowing the science that is behind it. Hopefully, current discussions will encourage more transparency and an exchange of NPS best practices. For now, I am hesitant to discuss the benefits and limitations of NPS on a more technical (i.e. statistical) level, but would like to do this in a future blog entry.
Posted by Alain on July 18, 2007
