Assuming a product, website, or app does what it intends to do and is usable, is there anything more?
Usability testing tends to focus on the objective task-oriented performance quality of an experience. What about aspects such as innovativeness, originality, or beauty? Do these matter? If so, how should they be measured?
We recently wrote about the evolution of three branches of standardized UX measurement from the 1970s to the present.
One of the forces driving the evolution of one major branch is the distinction between pragmatic and hedonic usability. In this article, we dive a little deeper into that distinction and discuss how the concepts of pragmatic and hedonic usability have themselves evolved over the past few decades.
What Is Hedonic Usability?
Before 2000, the industrial practitioners who worked on the development of products/systems for human use focused primarily on classical usability, assessing the extent to which designs led to successful and rapid task completion and high levels of satisfaction or perceived usability.
In the early 2000s, Marc Hassenzahl felt that the definition of usability, while broad, neglected the contribution of fun and enjoyment. He proposed a “hedonic” quality that includes non-task-oriented aspects such as innovativeness, originality, and the subjective nature of “appealingness.”
In 2000 and 2001, Marc Hassenzahl and colleagues published research on a distinction between the concepts of pragmatic (classical) usability and hedonic usability, defined by a set of seven-point semantic differential items (such as interesting-boring and impressive-nondescript) collected concurrently with a measure of appealingness (APPEAL) so they could investigate the extent to which pragmatic and hedonic usability accounted for variation in the measurement of APPEAL. Table 1 shows the original set of bipolar adjectives for these constructs (the AttrakDiff1 questionnaire).
|Construct||Item||Anchor 1||Anchor 2|
Different terms are used in the literature for pragmatic and hedonic usability
As research programs move forward and branch along different paths, associated terminology is often not perfectly consistent. Although the most common current terms for these constructs are pragmatic and hedonic usability, they are also referred to as
Pragmatic usability: pragmatic quality, ergonomic quality, perceived usability, task-oriented quality, instrumental product quality
Hedonic usability: hedonic quality, non-task-oriented quality, non-instrumental quality
Factor analysis supports the distinction between pragmatic and hedonic usability
In their 2000 paper, Hassenzahl et al. found evidence from factor analysis that the items in Table 1 aligned as expected with independent factors. Reliability for all three scales was high (coefficient alpha greater than .90). Scales derived from both the pragmatic and the hedonic items were significantly predictive of APPEAL, suggesting that appealingness of a product is affected by both pragmatic and hedonic aspects of user experience. The results of the 2001 paper were similar to the original findings. Alignment of items with factors generally was as expected and had good scale reliability, and there was almost equal contribution of pragmatic and hedonic measures to the prediction of product appealingness. Furthermore, concurrent collection of SMEQ (Subjective Mental Effort Questionnaire) data revealed a significant correlation with pragmatic usability, but no significant correlation with hedonic usability—evidence of appropriate convergent and divergent validity for the measures.
The hedonic construct has multidimensional relationships with perceived beauty and goodness
In response to other research into the relationship between perceived usability and beauty, Hassenzahl (2004) studied the relationship of usability (pragmatic and hedonic) with perceptions of beauty and goodness, concluding that perceptions of goodness were driven by both pragmatic and hedonic attributes, especially after use. Perception of beauty, on the other hand, depended on a “hedonic attribute group” named Identification, associated with the way products communicate important personal values to relevant others. Other refinements of hedonic quality include evocation (extent to which a product evokes memories) and stimulation (extent to which a product is perceived as challenging or novel).
How Do You Measure Hedonic Usability?
The most common ways of measuring hedonic usability are with the AttrakDiff2, the UEQ, and the meCUE2.0 standardized questionnaires.
The AttrakDiff questionnaire
Marc Hassenzahl developed the AttrakDiff questionnaire (Table 1) to differentiate between pragmatic and hedonic usability, with an emphasis on beauty and goodness. The current version is the AttrakDiff2, which consists of 28 seven-point semantic differential items to measure:
- Pragmatic usability (seven items)
- Hedonic identification (seven items)
- Hedonic stimulation (seven items)
- Attractiveness (formerly APPEAL) (seven items)
The items of the AttrakDiff2, shown in Table 2, are similar but not identical to those of the AttrakDiff1 questionnaire shown in Table 1 (see the “Same as V1” column). The seven response options for AttrakDiff2 range from -3 to +3, with 0 at the center of the scale.
|Construct||Item||Anchor 1||Anchor 2||Same as V1|
|6||separates me||brings me closer|
The User Experience Questionnaire (UEQ)
Like the AttrakDiff, the UEQ (User Experience Questionnaire, Laugwitz et al., 2008; see Table 3) assesses pragmatic and hedonic usability with seven-point semantic differential items, using 26 items to measure
- Perspicuity (4 items)
- Efficiency (4 items)
- Dependability (4 items)
- Novelty (4 items)
- Stimulation (4 items)
- Attractiveness (6 items)
Response options range from -3 to +3. Seven UEQ items have the same anchors as the AttrakDiff2; four have one anchor in common. The location of the common items is reasonably consistent with AttrakDiff2, with a few exceptions. For example, dull-captivating and unimaginative-creative are part of AttrakDiff2’s Hedonic-Stimulation construct, while creative-dull is an item in UEQ’s Hedonic-Novelty construct. Discouraging-motivating is a pair in AttrakDiff2’s Attractiveness construct; motivating-demotivating is in UEQ’s Hedonic-Stimulation construct.
|Construct||Item||Anchor 1||Anchor 2||In AttrakDiff2?|
|Pragmatic -||1||not understandable||understandable|
|Perspicuity||2||easy to learn||difficult to learn|
|4||meets expectation||does not meet expectation|
The Components of User Experience (modular evaluation) meCUE questionnaire
The meCUE is a standardized questionnaire based on Thüring and Mahlke’s (2007)’s Components of User Experience model. Its latest version is the meCUE2.0 with 33 items. It’s designed to provide a modular evaluation (that’s the “me”) with each of its components:
- Perceived usefulness (2 items)
- Perceived usability (3 items)
- Aesthetics (3 items)
- Status (3 items)
- Commitment (3 items)
- Positive emotions (6 items)
- Negative emotions (6 items)
- Intention to use (3 items)
- Loyalty (3 items)
- Overall experience (1 item)
The meCUE 2.0 is the longest of the three questionnaires and shares some aspects (in addition to pragmatic usability) with other standardized UX questionnaires (e.g., usefulness with the TAM/UMUX-LITE and loyalty with the SUPR-Q).
The meCUE 2.0 items use a seven-point agreement scale except for the rating of overall experience, which is a 21-point bipolar scale anchored with “as bad” and “as good,” with response options from -5 to +5 in half-point increments. The seven-point scales are fully labeled without numbers using the following response option labels: strongly disagree, disagree, somewhat disagree, neither agree nor disagree, somewhat agree, agree, strongly agree. Table 4 shows the wording of the meCUE2.0 items and their associated components.
|I||Usefulness||The functions of the product are exactly right for my goals.|
|It is quickly apparent how to use the product.|
|The operating procedures of the product are simple to understand.|
|Usability||The product is easy to use.|
|I consider the product extremely useful.|
|With the help of this product I will achieve my goals.|
|II||Visual aesthetics||The product is creatively designed.|
|The design looks attractive.|
|The product is stylish.|
|Status||The product would enhance my standing among my peers.|
|By using this product, I would be perceived differently.|
|I would not mind if my friends envied me for this product.|
|Commitment||I could not live without this product.|
|This product is like a friend to me.|
|If I ever lost the product, I would be devastated.|
|III||Positive emotions||The product exhilarates me.|
|The product relaxes me.|
|The product makes me feel happy.|
|The product makes me feel euphoric.|
|The product calms me.|
|When using this product, I feel cheerful.|
|Negative emotions||The product makes me tired.|
|The product annoys me.|
|When using this product I feel exhausted.|
|The product frustrates me.|
|The product makes me feel passive.|
|The product angers me.|
|IV||Intention to use||If I could, I would use the product daily.|
|I can hardly wait to use the product again.|
|When using the product, I lose track of time.|
|Product loyalty||I would not swap this product for any other.|
|In comparison to this product, no others come close.|
|I would get exactly this product for myself (again) at any time.|
|V||Overall evaluation||How do you experience the product as a whole?|
Future of Hedonic Usability
The concept of hedonic usability is reasonably compelling, and the research supporting it seems solid. For practical UX work, however, there are some issues with current approaches to its measurement that stand in the way of wide adoption.
Length, lack of global scores, and absence of norms inhibit adoption
Inhibitors to the use of questionnaires such as AttrakDiff2, UEQ, and meCUE2.0 include
- Length: In their standard forms, there are from 26 to 33 items in these questionnaires. The short forms that have been developed for AttrakDiff2 and UEQ [pdf] may help their adoption.
- No global scores: For theoretical reasons, the developers of these questionnaires made the design decision to avoid providing global scores. This can make these instruments difficult to use by practitioners who need to make a global conclusion about product quality.
- Absence of norms: Except for UEQ, there are no published norms for interpreting scores. The work that the UEQ developers have put into developing interpretive norms may pay off in greater future adoption.
Additional concurrent analysis with the SUS and UMUX-LITE needed
Both the SUS and the UMUX-LITE have well-established norms for measuring their perceived usability. It’s possible that one or more constructs in the class of pragmatic-hedonic questionnaires might not just correlate with SUS or UMUX-LITE but might correspond in magnitude well enough to be able to use their open-source norms. There would also be value in systematically assessing how well this class of questionnaires, along with SUS/UMUX-LITE, accounts for variability in outcome metrics such as ratings of overall experience and likelihood-to-recommend. This would help address the question of what hedonic usability measures capture that pragmatic usability measures don’t.
Pragmatic usability focuses on the task-oriented nature of an experience, whereas hedonic usability focuses more on the appeal, fun, and originality. The concept of hedonic usability is about 20 years old. Over those 20 years, the measurement of hedonic usability has expanded to multiple constructs and is measured using different questionnaires, most commonly AttrakDiff2, UEQ, and meCUE2.0. Based on the content of their items and structural analyses, all three appear to be viable instruments for UX research; although they are all lengthy (26–33 items), which may inhibit their usage. Only the UEQ has benchmarks for the interpretation of its scores, which is of particular value to practitioners.
Learn More: UX Measurement Boot Camp
Intensive Training on UX Methods, Metrics and Measurement
|Denver: Aug. 5th-7th, 2020|