A Longitudinal Examination of Net Promoter and Firm Revenue Growth

In the study, the authors use a ten-point version of the NPS (1 to 10), which is similar, although not identical to the Reichheld version. They asked “How likely or unlikely is it that you would recommend of the question if a friend or business relation asked for your advice” and used the end-point labels “very high probability/very low probability.”

They examined 17 Norwegian firms across five industries and correlated the scores along with several other metrics (including the NCSB, which is similar to the ACSI) to change in revenue. Each industry had a minimum of three firms (the minimum number needed for correlations), which represents a small sample size and makes correlations quite erratic as we found in our earlier industry analysis.

For banking, the pooled correlation (partial for auto-correlation) was r = .40, which was the highest among other measures. For retail gasoline, the correlation was negative (r = -.45) but top-two-box satisfaction was the strongest (r = .84). For retail home furnishings the correlation was negative (r = -.12) and top-box satisfaction was the highest (r =.99). For security systems, the correlation was r = .86, behind top-two-box recommendation intention (essentially promoters). For transportation, few measures correlated, with the NPS correlating at r = .08, just behind the leader, which is repurchase intention r= .14. In short, they found the NPS was the best or second best predictor in two of the five industries (17 companies total) in the years they studied. However, given the small sample sizes within each industry (between three and five), even large differences in correlations were not statistically different.

Keiningham and colleagues then revisited the NPS data reported in Reichheld (2003 and 2006) and showed that the R-square values for three industries were comparable (and statistically indistinguishable) using both the ACSI and NPS. They argued that “no researchers have attempted to replicate the research methodology” although they do cite the Marsden et al. (2005) study and the flawed work by Morgan and Rego (2006).

Keiningham et al.’s 2007 NCSB analysis and NPS vs. ACSI reconstruction show that the NPS is not always a CLEAR winner. However, both datasets used have insufficient power by themselves to dismiss the NPS. Just to compute a confidence interval around a correlation requires at least four data points (because the correlation formula has three degrees of freedom in the denominator)—See Quantifying the User Experience (p. 294). However, a similar concern can be levied against Reichheld’s original work, which also had only a few companies in each industry. The correlations reported by Keningham et al. however can be used in a meta-analysis by combining them with others studies.

This paper received the Marketing Science Institute/H. Paul Root Award. (See their earlier paper from 2007).

Another version of the same paper was published again under the title “A Holistic Examination of Net Promoter” (Keiningham et al., 2008) in a different journal, so be sure not to treat these as independent sources.

Takeaway: Their analysis shows that NPS correlates highly with other metrics, most notably customer satisfaction, and sometimes is the best, but not always. This does contradict Reichheld’s’ assertion that satisfaction doesn’t correlate with growth, but similar to our analysis and others, it shows satisfaction and NPS provide comparable predictive abilities. This paper is a good example of generally what other papers show: NPS isn’t always clearly the best predictor of growth metrics, but even when it’s not, it’s a close contender.