In an earlier article, we examined the relationship between NPS and future company growth.
We found the Net Promoter Score had a modest correlation with future growth (r = .35) in the 14 industries we examined.
In the 11 industries that had a positive correlation, the average correlation with two-year revenue growth was higher at r = .44. This ranged from a high of r = .89 for U.S. airlines, r = .49 for retailers, and r = .50 for investment banking to a low of r = .22 for grocery stores and r = .26 for wireless providers.
In other words, we found reasonable evidence for the predictive validity of the Net Promoter Score. However, one claim made by Reichheld was that the NPS is a better predictor of growth than customer satisfaction.
In our earlier analysis, we reviewed the published literature and conducted an analysis with U.S. airlines. We found little evidence in the literature and in our analysis to suggest that the NPS was superior to customer satisfaction. In most cases it performed the same as satisfaction, but not always—and sometimes it was worse.
In this article, we will revisit the dataset we used to examine the predictive validity of the Net Promoter Score and see how well a common measure of satisfaction performed.
Satisfaction and NPS Data Correlated to Future Growth
For business growth data, we used the same growth metrics used in the NPS analysis. It contained data from 149 companies in 14 industries from 2013 to 2017. For satisfaction data, we used the American Customer Satisfaction Index (ACSI), which has been cited in earlier research and is one of the most popular measures of satisfaction. The ACSI uses three questions (overall satisfaction, meeting expectations, and comparison to an ideal company) on three ten-point scales. The responses are then weighted using a proprietary formula, although it’s likely an unweighted average of the three would correlate very highly—see Nunnally (1994) for a discussion on weights.
We were able to obtain 2013 ACSI benchmarks for 12 of the 14 industries we used in the NPS study. The benchmarks are available online. We were unable to find sufficient data for two industries we used in the NPS analysis: the software and parcel industries. The ACSI only tracked Microsoft, whereas we had NPS data for Google, Activision, Microsoft, Symantec, Adobe, Intuit, McAfee, and Blackboard. We technically need a minimum of two data points to generate a correlation between attitudinal measures and future growth, but ideally we need three to have some range in the correlation.
Even with industries where we had enough coverage, there were still differences in which companies ACSI tracked and the companies tracked in the Temkin NPS report. For example, in the banking industry, we had 2013 NPS data on 14 U.S. banks but only 4 banks from ACSI (Citibank, Chase, Wells Fargo, and Bank of America). For the airline industry, we had better overlap—NPS data for five airlines and ACSI data for four airlines for 2013. There were around half as many companies tracked in the ACSI compared to the NPS report we used (149 for NPS and 74 for ACSI) resulting in 71 companies in common.
Table 1 shows the 14 industries, the metric we used, the number of companies by industry used for the NPS analysis, the number we obtained for the ACSI, and the number in common (data on NPS, ACSI, and growth data) by industry.
|Industry||Metric||# of NPS Companies||# of ACSI Companies||# In Common|
|Grocery||Same store sales||7||4||3|
|Auto dealers||Car sales (numbers)||19||20||18|
|Fast food||Same store sales||11||8||8|
|Retailers||Same store sales||31||8||8|
|Health plans||Premium revenue||10||4||4|
First, we looked at the average correlations by industry for the NPS and the ACSI. Keep in mind there are different companies used (mostly based on limited availability from the ACSI data). Table 2 shows the correlations by industry, the overall average correlation, and the correlation between the NPS and the ACSI by industry (for the 71 companies in common).
|Industry||NPS||ACSI||ACSI NPS r|
On average, the correlation between the NPS and two-year future growth was nominally, but not statistically, higher (r = .35) than the ACSI correlation (r = .28). The NPS had higher correlation in 7 of the 12 industries (almost a tie with ACSI), suggesting the ACSI was better in about half but the NPS was better in about half. For example, the correlation for the ACSI and future net income growth in the banking industry is a negative r = -.40 but r = .42 for the NPS. In contrast, the correlation between the NPS and premium growth in the insurance industry is r = .31 but a higher r = .75 for the ACSI.
The far right column in Table 2 also shows the correlation between the NPS and the ACSI by company. Overall, the average correlation is a strong r = .83 (after the Fisher transformation), showing that satisfaction and intention to recommend are closely related. The sample sizes were small for some industries (e.g., grocery stores had three companies in common), so it’s not too surprising to see some large swings in the correlation. Despite the small sample sizes within each industry, only one industry (health plans) had a negative correlation between the NPS and the ACSI.
Comparing Using the Same Companies
Part of the difference between correlations is because of the different companies used for the NPS and the ACSI, making it potentially difficult to differentiate between real differences in predictive ability and merely different sample characteristics. To help make a more “apples to apples” comparison we next only included companies that had both NPS and ACSI data (71 companies). Table 3 shows the results of these correlations for the 12 industries.
|NPS 2013||ACSI 2013|
Again, we see similar results using the same set of companies but the differences in the aggregated correlations were larger. The average correlation between the 2013 NPS data and 2013–2015 growth is r = .40 compared to r =.23 for the ACSI data. In this case, half the industries have higher correlations with the NPS than the ACSI. For example, the PC industry has a correlation of r = .39 for NPS and r = .57 for the ACSI.
As we did in our earlier analysis, we also extended the correlation to a longer four-year growth period (2013–2017) as shown in Table 4.
|NPS 2013||ACSI 2013|
We still see similar results using the same set of companies for the future four-year period (2013–2017). The average correlation between the 2013 NPS data and 2013–2017 growth is r = .38 compared to r =.13 for the ACSI data. In this case, seven of the industries have higher correlations with the NPS than the ACSI. For example, the NPS has a positive correlation of r = .26 in the wireless industry, compared to a negative r = -.61 correlation with the ASCI.
However, counting the relative standing of each correlation may not be a good way to differentiate between the predictive abilities. For example, while the NPS had a correlation that was slightly better than the ACSI in the PC industry, both were negatively correlated (-.58 and -.64 respectively), suggesting neither was a good predictor for that industry during that period (using our growth metrics and the companies we had data on).
It’s interesting how well both the NPS and the ACSI predicted the future growth in the airline industry. Even after four years, the correlation was extremely strong at r = .92 and r = .97 respectively. This corroborates our earlier findings with the U.S. airline industry, which showed that both the NPS and satisfaction were good predictors, but the ACSI was nominally better.
Discussion and Takeaways
In our analysis of NPS and satisfaction data from the ACSI we found:
The NPS correlates nominally higher than the ACSI with future growth. The average correlation between the NPS and future growth metrics in 12 industries was higher (but not statistically) than the ACSI using all three measures. For two-year future growth, the NPS average correlation was r = .35 compared to r =. 28 for ACSI (using different companies). For two-year future growth using the same 71 companies, the average correlation was r = .40 compared to r =. 23 for the ACSI. For four-year future growth using the same 71 companies, the average correlation was r = .38 compared to r =. 13 for the ACSI. The differences are not statistically significant, which is expected given the relatively small sample size of 12 industries. However, these correlations can be used as part of a meta-analytic view of other correlations between satisfaction, the NPS, and growth metrics.
The differentiation of the extremes may help. One possible reason for the slightly better performance of the NPS over the ACSI with this dataset could be how the NPS accounts for the extreme responders—both extremely likely to recommend (promoters, top-two box) and those less likely (detractors, or bottom-six box). The ACSI uses a proprietary weighted combination of three questions that we suspect relies on the mean, instead of extreme, responders. For example, Jones & Sasser (1995) showed that a top-box, extremely satisfied, customer at Xerox was six times more likely than a merely satisfied customer to repurchase Xerox products over the next 18 months. Similarly, de Haan et al. (2015) found changes in top-two box customer satisfaction, followed by the official NPS, had the highest impact on customer retention.
It’s behavioral intention vs. satisfaction. Another reason for why the NPS correlations with growth were nominally higher is because the NPS is a measure of behavioral intention (likelihood to recommend). In contrast, the ACSI is a measure of attitude. While attitudes do predict behavior, they tend to predict behavior through behavioral intentions. This idea is also supported by Pollack & Alexandrov (2013), who argue that satisfaction can be seen as an antecedent to repurchase intentions.
Other satisfaction measures may correlate better. We only examined one source of satisfaction data (although it’s one of the most popular). It’s possible that another measure of satisfaction, for example, from JD Power, or from company-owned satisfaction data, may correlate better than the NPS. It’s something to be examined in a future analysis.
It qualifies earlier research of satisfaction vs. the NPS. Some earlier studies have shown the NPS generated similar results as satisfaction. In cases where the NPS didn’t correlate as strongly, it was close. This suggests that the NPS may not be necessarily a superior predictor of growth (although it appears to be AT LEAST as good in this analysis), but it may be another comparable predictor. For example, an earlier study by Keiningham et al. (2007) found no correlation between the NPS and growth in five Norwegian industries (e.g., gas stations) but also found no correlation between customer satisfaction and growth. In the same paper, they found the ACSI had almost the same correlation with historical growth as the NPS (again showing that measures of satisfaction and the NPS are likely correlated: when one correlates with growth, they both do). The strong correlation between the NPS and the ACSI (r = .83) in this analysis further supports this relationship between satisfaction and intent to recommend.
Correlations differ by industry. For some industries neither the NPS nor the ACSI were good predictors of future growth (at least using our metrics during this period). However, for some industries, both were solid predictors of growth, such as the U.S. airline industry where even after four years, both the NPS and the ACSI had correlations above r = .90. This corroborates our earlier findings with the U.S. airline industry, which showed that both were good predictors but the ACSI was nominally better. This is also echoed in Pollack & Alexandrov (2013) who caution that the relationship between the NPS and satisfaction metrics is neither simple nor linear, and it depends on the industry.
Replication is needed. Collecting growth data for each industry requires making decisions about which business metric is a better measure of growth. For some industries this was simply total revenue; for others, it was growth in same store sales (retail) or asset growth (banking). See the earlier article for list of sources for growth data, and the NPS and ACSI data can be easily downloaded. Even with an “objective” intention, biases can creep into an analysis and bend the results. We have documented our assumptions for others to replicate (or qualify).
Learn More: UX Measurement Boot Camp
Intensive Training on UX Methods, Metrics and Measurement
|Fall 2020: Delivered Online|