In this paper, the authors discuss the importance of using a relative satisfaction measure to predict a measure called Share of Wallet (SOW). SOW is how people allocate their spending within a category, say between two or three grocery stores.
The authors conducted a longitudinal study across six months with around 80,000 customers in 15 countries. The subject of the analysis was based only on the participants who answered the follow-up questions six months later. This reduced the final count to 1,138 US customers answering satisfaction questions about grocery stores, airlines, drugstores, and several others.
Respondents used a ten-point satisfaction scale (1 = completely dissatisfied and 10 = completely satisfied), and raw satisfaction scores were converted to ranks. For example, if a respondent rated Brand A 7 on satisfaction and Brand B 5, the ratings become 1 and 2. The authors here use a single-item measure and argue that satisfaction is “concrete and singular in nature.” Note that in 2008 Keiningham et al. criticized the NPS for using a single item. Interestingly, they state in this 2015 article, “Moreover, psychometric analyses conducted by Drolet and Morrison (2001) finds that the incremental information from even the second or third item in a multi-item scale contributes very little to the information obtained from the first item in a multi-item scale.”
Many authors report that single-item measures perform adequately (e.g., Bolton, 1998; Bolton & Lemon, 1999; Cooil et al., 2007; Crosby & Stephens, 1987; Drolet & Morrison, 2001; Mittal & Kamakura, 2001; Mittal et al., 1998 and 1999). Most firms use single-item measures of satisfaction (Morgan et al., 2005).
Drolet and Morrison, as reported in Keiningham et al.’s 2015 article, also found that “added items actually aggravate respondent behavior, inflating across-item error term correlation and undermining respondent reliability” (p. 196).
To confirm that their single satisfaction measure was adequate, Keiningham and colleagues correlated their single satisfaction item with a composite average of all three ACSI items (satisfaction, performance relative to expectations, and performance relative to the customer’s ideal) and found a very high correlation (r = .95).
The authors used several transformations of the change in satisfaction rankings to best predict the change in SOW. Their best-fitting transformation used a Zipf transformation and had a correlation of r = .403. The raw relative rank in satisfaction without the transformation had an r = .285. The raw satisfaction score had a low r = .066 with change in SOW as did recommend intention (r = .065) and the Net Promoter Score (r = .067).
The authors did not report how they collected the NPS or recommend intention; however, a clarification was provided by Bruce Cooil (personal communication, May 17, 2019) who stated that the NPS was computed using the Reichheld three-point classification of promoter, passive, and detractor and correlated highly with the other metrics. The authors firmly concluded that “changes in a customer’s Net Promoter classification similarly has almost no correlation to changes in share of wallet.”
However, they did not test the same transformations used on satisfaction, so the conclusion doesn’t follow the data. That is, the correlation with raw satisfaction to the change in the share of wallet was virtually identical to the Net Promoter Score (r= .066 vs r = .067). Given the high correlation shown in this paper and other studies, it’s likely the transformed NPS would provide comparable results to transformed satisfaction. This was something the reviewers should have caught.
The authors claimed that there is a wide body of scientific evidence casting doubt on the NPS link to business outcomes. They cited their earlier paper (Keiningham, 2007), Morgan and Rego (who have earlier been discounted by the authors), and the opinion piece by Sharp (2008).
Takeaways: The authors firmly concluded (as cited in the WSJ article) that “changes in a customer’s Net Promoter classification similarly has almost no correlation to changes in share of wallet.” However, the authors did not see whether transforming NPS as they did with satisfaction would have a higher correlation in predicting changes in SOW. This limits the generalizability of their (negative) claims about the NPS from this study.