The NPS and the ACSI: A Critique and an Alternative Metric
East, R., Romaniuk, J., & Lomax, W. (2011).
The authors compared the ACSI and NPS and argued that both have shortcomings as neither include ex-users and never-users of the brand, who are not questioned in the NPS survey. This is more of a criticism of whom to survey as opposed to what to ask people. East and colleagues take exception to Reichheld’s claim that “detractors are responsible for 80% to 90% of a company’s negative word of mouth.” In our analysis of negative comments, we did find that the bulk of negative comments from current users came from detractors.
To show that more negative word of mouth comes from noncustomers or never customers, East and colleagues conducted a convenience sample collecting data from 2,254 participants across ten studies in the UK and asked participants to recall the number of times they gave positive advice or received negative advice for a number of industries, including supermarkets, coffee shops, and skin care products. Participants also noted if they were current, former, or never users of the brand.
They found that most (71%) of the positive word of mouth came from current users and 77% of negative word of mouth came from former or never users. Across the three industries, they found promoters accounted for between 32–45% of positive word of mouth. Using a similar net scoring system with the ACSI, they found comparable results, with 26%–44% of positive word of mouth coming from promoters. However, they found a much smaller percent of negative comments came from detractors, between 3 and 31% for the NPS and 0 and 28% for the ACSI. They also found that NPS and ACSI detractors accounted for between 8 and 22% of positive comments.
This is comparable to our analysis on detractors, where we found only 21% of comments were negative about customers’ current brands. A low-scoring LTR doesn’t mean you will make negative comments, but we found that if you made a negative comment, you were much more likely to be a detractor than promoter. The authors did not limit the percentages to current customers as we did.
The authors compared a ten-item measure assessing positive and negative word of mouth which they call NEW (Net effect of Word of Mouth) with ACSI and NPS. The measure is similar to a measure suggested by Samson (2006).
They reported very high correlations between ACSI and NPS (r = .99, .74 and .92 for each industry). The correlation with the NEW metrics was very high for supermarkets (r = .96, r = .98 with the NPS and ACSI) but lower for coffee shops and skin care products (r = .19 to .34 , all non significant). The authors see this lack of correspondence with the NPS and ACSI as a promising indication of discriminant validity (it’s a different measure) and a subject for future work.
The authors also confirm the high correlation between NPS and ACSI and they checked the correlations at the individual level as opposed to aggregated at the company level and found relatively high correlations (r = .55).
Takeaway: The authors show that the NPS and ACSI are highly correlated (r = .99, .74 and .92) but take exception to the NPS’s accounting for negative comments. They did corroborate Reichheld’s finding that promoters accounted for the largest share of positive comments. But they found that most negative comments about a brand come from former customers or never customers. In short, their criticism of the NPS was applied to the ACSI and would apply to any multi-item measure that asks only current customers. So their main criticism here is a suggestion to sample more than current customers to better gauge negative WOM.