While customer satisfaction may be thought of as one concept, there’s isn’t a single “official” way to measure it.
By one estimate there are more than 40 instances of different customer satisfaction scales described in the published literature.
That, in part, is a consequence of how common satisfaction is as a measure. Satisfaction is measured on more than just brands, products, and features. It’s used to gauge one’s attitude toward one’s job, life, public service, and the quality of a relationship to name a few.
Satisfaction can be measured using either multiple items in a questionnaire or only a single item scale. And it’s the format of the scales themselves that is the subject of this article.
Across much of the published literature, satisfaction scales fall into three broad categories and subcategories:
- Satisfaction scales
- Unipolar rating
- Unipolar thermometer
- Bipolar rating
- Performance scales
- Disconfirmation scales
Somewhat confusingly, satisfaction scales also have a subgroup called satisfaction scales. I’ve broken this group into three further subgroups (unipolar rating scales, unipolar thermometers, and bipolar rating scales) as some research suggests that they have different reliability and discriminating characteristics. That gives us five common ways you’ll see satisfaction measured, with some notes about how particular scales have performed in published research. All variations of these scales can be used in our MUIQ research platform.
- Unipolar Scales
When you think of satisfaction scales, unipolar scales are probably what you have in mind. In the unipolar scale, only satisfaction is considered (not dissatisfaction), and it’s typically presented on a continuum (sometimes fully labeled) from completely satisfied to not at all satisfied. These unipolar satisfaction scales typically have between 2 and 101 points or use a graphical scale with faces or a line or slider for participants to mark. Below is an example of a five-point fully labeled (no-numbers) version from Aiello and Czepiel (1979).
Unipolar scales can also be labeled on just the endpoints (a topic for a future article), such as the one below. With unipolar satisfaction scales you’re using only one concept — satisfaction — and aren’t including an opposite (usually dissatisfaction).
One possible advantage to using only satisfaction (and not dissatisfaction) is you avoid having a clear opposite item. As simple as it may sound, dissatisfaction may not be interpreted as the opposite of satisfaction by your respondents (depending on the context). If it’s not the opposite it may add more error (less reliability) in responses as people may interpret midpoints differently.
- Unipolar Feeling Thermometer
A special case of the unipolar rating scale is something that sort of acts like a satisfaction thermometer. Respondents are asked to select the percent satisfied they are, from not at all satisfied (0% satisfied) to completely satisfied (100% satisfied), which takes up 11 points on the scale.
For example, a study by Westbrook (1980) used an 11-point feeling thermometer with completely satisfied on the left. A more recent study by Maritz (2011) [pdf] used a web-based version and found that their unipolar feeling thermometer performed best relative to other satisfaction scales. Feeling thermometers can use fewer points (e.g., six points with 20% point increments), but with fewer points of discrimination they would likely be slightly less reliable and offer less discriminating capability.
- Bipolar Satisfaction
While looking quite similar, the bipolar satisfaction scale uses the putative opposites on the ends (poles) of the rating scales. This is usually an extreme version of satisfaction at one end and an extreme version of dissatisfaction at the other. While it seems that dissatisfaction is the natural opposite of satisfaction, there’s some evidence [pdf] that people treat dissatisfying events differently and that such events play an equally important (if not more important) role in loyalty. If dissatisfaction isn’t the opposite, then it also makes the neutral point more problematic.
Bipolar satisfaction scales don’t need to be anchored by only satisfaction and dissatisfaction. In a study by Westbrook in 1980, he used a Delighted to Terrible scale (shown below) and found that it had nominally higher reliability relative to other satisfaction scales he tested. Those results are interesting considering the study’s quite varied use of root words (satisfied, delighted, unhappy, terrible, and pleased).
There is some evidence [pdf] satisfaction includes both hedonic and utilitarian qualities. The Delighted to Terrible scale anchors may tap into the more hedonic qualities of satisfaction whereas the unipolar thermometers may be more utilitarian.
A satisfaction performance scale asks participants to consider the adequacy/performance of a product or experience. It can be presented as an overall judgment or more specifically about attributes such as the following four-point scale adapted from a self-reported health rating scale.
Performance scales can also be used to gauge the quality of an experience. For example, they can be used to rate the overall quality of a hotel and more specifically the room, check-in experience, and hotel restaurant. For rating product performance this could be about a mobile phone overall and specifically about the battery life, camera quality, and user interface. Here are examples of two bipolar performance scales adapted from Churchill and Surprenant (1982).
A likely less familiar but still important way to measure satisfaction is to understand how well expectations were met (confirmed) or not met (disconfirmed). Measures of disconfirmation tap into a simple idea of assessing how well a product or experience was better or worse than expected, such as the example below.
There is some evidence that these disconfirmation scales may be a better measure of failing to meet expectations than comparing pre- and post-expectation measures.
And other research has also found that these disconfirmation scales may better correlate with customer retention than satisfaction or performance scales. (See Rust et al. 1994.)
Discussion and Takeaway
A review of the literature on measuring customer satisfaction revealed:
There are three common satisfaction scale types. It may be helpful to think of satisfaction scales as falling into three broad types: satisfaction scales, performance scales, and disconfirmation scales. Satisfaction scales are likely the most familiar to researchers and can be subdivided into unipolar rating scales, unipolar thermometers, and bipolar rating scales (providing five satisfaction scale types). Other factors will affect the reliability, validity, and sensitivity of satisfaction scales, including orientation (positive vs. negative options first), labeling (full vs. partial) and the number of scale points. This is the subject of many past and future articles.
Satisfaction may be multi-dimensional. While satisfaction may be thought of as a single construct, and in some contexts may be unidimensional, there is also evidence it has multiple dimensions in other contexts. Possible additional dimensions of satisfaction include dissatisfaction, hedonic vs. utilitarian satisfaction, performance, and disconfirmation. One emerging model is that performance drives disconfirmation, which in turn drives satisfaction. (See Oliver 1993 and Danaher and Haddrell 1996.)
Multiple measures may be better but longer. So which scale(s) do you use? The literature provides mixed results on which type of satisfaction scale is “best” and some studies — e.g., Churchill and Surprenant (1982) — suggest a combination of all three can provide a better picture of satisfaction. From a practical standpoint, however, researchers are often limited in the number of questions they can ask (especially about the same construct). So, while using more items will likely increase the reliability and validity of your scale (especially if satisfaction is multidimensional), it may be modest and not enough to offset the added load of more questions. If you are able, there’s little harm in asking all three types of satisfaction questions and then comparing results (and look for the connection between these complementary types of satisfaction).
There is no universal “best scale.” The published research provides mixed results on the quality of each of these satisfaction scales and which one is “best.” The unclear winner is most likely due to two main reasons. First, the scales are evaluated in often very different contexts, with different types of participants gauging things from consumer electronics, flu vaccines, restaurants, and hotels to name a few. That context suggests some scales (e.g., performance scales) may be more appropriate in some situations but not in others. The second reason is that the actual scales vary in their presentation style and confound the effects. The number of points, the labels used, whether they’re fully or partially labeled, the number of items, and the orientation (positive vs. negative first) make disentangling the effects more challenging.
Research into satisfaction is broad and ongoing. Some customer satisfaction articles I reviewed for this article date back fifty years, with many of the more influential papers appearing in the 1980s. My hope is that by delineating the distinction between the most common types of scales and knowing the relevant studies, other researchers can better select the right scale and address potential shortcomings or see how these different aspects complement each other to provide a better picture of satisfaction, loyalty, and future consumer behavior.
Learn More: UX Measurement Boot Camp
Intensive Training on UX Methods, Metrics and Measurement
|Denver: Aug. 5th-7th, 2020|