Is a usable experience sufficient for a good experience?

Assuming a product, website, or app does what it intends to do and is usable, is there anything more?

Usability testing tends to focus on the objective task-oriented performance quality of an experience. What about aspects such as innovativeness, originality, or beauty? Do these matter? If so, how should they be measured?

We recently wrote about the evolution of three branches of standardized UX measurement from the 1970s to the present.

One of the forces driving the evolution of one major branch is the distinction between pragmatic and hedonic usability. In this article, we dive a little deeper into that distinction and discuss how the concepts of pragmatic and hedonic usability have themselves evolved over the past few decades.

What Is Hedonic Usability?

Before 2000, the industrial practitioners who worked on the development of products/systems for human use focused primarily on classical usability, assessing the extent to which designs led to successful and rapid task completion and high levels of satisfaction or perceived usability.

In the early 2000s, Marc Hassenzahl felt that the definition of usability, while broad, neglected the contribution of fun and enjoyment. He proposed a “hedonic” quality that includes non-task-oriented aspects such as innovativeness, originality, and the subjective nature of “appealingness.”

In 2000 and 2001, Marc Hassenzahl and colleagues published research on a distinction between the concepts of pragmatic (classical) usability and hedonic usability, defined by a set of seven-point semantic differential items (such as interesting-boring and impressive-nondescript) collected concurrently with a measure of appealingness (APPEAL) so they could investigate the extent to which pragmatic and hedonic usability accounted for variation in the measurement of APPEAL. Table 1 shows the original set of bipolar adjectives for these constructs (the AttrakDiff1 questionnaire).

ConstructItemAnchor 1Anchor 2
Pragmatic1comprehensibleincomprehensible
2supportingobstructing
3simplecomplex
4predictableunpredictable
5clearconfusing
6trustworthyshady
7controllableuncontrollable
8familiarstrange
Hedonic1interestingboring
2costlycheap
3excitingdull
4exclusivestandard
5impressivenondescript
6originalordinary
7innovativeconservative
APPEAL1pleasantunpleasant
2goodbad
3aestheticunaesthetic
4invitingrejecting
5attractiveunattractive
6sympatheticunsympathetic
7motivatingdiscouraging
8desirableundesirable

Table 1: AttrakDiff1 items for Pragmatic (Ergonomic), Hedonic, and APPEAL constructs.

Different terms are used in the literature for pragmatic and hedonic usability

As research programs move forward and branch along different paths, associated terminology is often not perfectly consistent. Although the most common current terms for these constructs are pragmatic and hedonic usability, they are also referred to as

Pragmatic usability: pragmatic quality, ergonomic quality, perceived usability, task-oriented quality, instrumental product quality

Hedonic usability: hedonic quality, non-task-oriented quality, non-instrumental quality

Factor analysis supports the distinction between pragmatic and hedonic usability

In their 2000 paper, Hassenzahl et al. found evidence from factor analysis that the items in Table 1 aligned as expected with independent factors. Reliability for all three scales was high (coefficient alpha greater than .90). Scales derived from both the pragmatic and the hedonic items were significantly predictive of APPEAL, suggesting that appealingness of a product is affected by both pragmatic and hedonic aspects of user experience. The results of the 2001 paper were similar to the original findings. Alignment of items with factors generally was as expected and had good scale reliability, and there was almost equal contribution of pragmatic and hedonic measures to the prediction of product appealingness. Furthermore, concurrent collection of SMEQ (Subjective Mental Effort Questionnaire) data revealed a significant correlation with pragmatic usability, but no significant correlation with hedonic usability—evidence of appropriate convergent and divergent validity for the measures.

The hedonic construct has multidimensional relationships with perceived beauty and goodness

In response to other research into the relationship between perceived usability and beauty, Hassenzahl (2004) studied the relationship of usability (pragmatic and hedonic) with perceptions of beauty and goodness, concluding that perceptions of goodness were driven by both pragmatic and hedonic attributes, especially after use. Perception of beauty, on the other hand, depended on a “hedonic attribute group” named Identification, associated with the way products communicate important personal values to relevant others. Other refinements of hedonic quality include evocation (extent to which a product evokes memories) and stimulation (extent to which a product is perceived as challenging or novel).

How Do You Measure Hedonic Usability?

The most common ways of measuring hedonic usability are with the AttrakDiff2, the UEQ, and the meCUE2.0 standardized questionnaires.

The AttrakDiff questionnaire

Marc Hassenzahl developed the AttrakDiff questionnaire (Table 1) to differentiate between pragmatic and hedonic usability, with an emphasis on beauty and goodness. The current version is the AttrakDiff2, which consists of 28 seven-point semantic differential items to measure:

  • Pragmatic usability (seven items)
  • Hedonic identification (seven items)
  • Hedonic stimulation (seven items)
  • Attractiveness (formerly APPEAL) (seven items)

The items of the AttrakDiff2, shown in Table 2, are similar but not identical to those of the AttrakDiff1 questionnaire shown in Table 1 (see the “Same as V1” column). The seven response options for AttrakDiff2 range from -3 to +3, with 0 at the center of the scale.

ConstructItemAnchor 1Anchor 2Same as V1
Pragmatic1technicalhuman
2complicatedsimplePartial
3impracticalpractical
4cumbersomestraightforward
5unpredictablepredictableYes
6confusingclearly structuredPartial
7unrulymanageable
Hedonic - 1isolatingconnective
Identification2unprofessionalprofessional
3tackystylish
4cheappremiumPartial
5alienatingIntegrating
6separates mebrings me closer
7unpresentablepresentable
Hedonic - 1conventional inventive
Stimulation2unimaginativecreative
3cautiousbold
4conservative innovativeYes
5dullcaptivatingPartial
6undemandingchallenging
7ordinarynovelPartial
Attractiveness1unpleasantpleasantYes
2uglyattractivePartial
3disagreeablelikeable
4rejectinginvitingYes
5badgoodYes
6repellingappealing
7discouragingmotivatingYes

Table 2: AttrakDiff2 items—Yes in the “Same” column indicates the adjective pair was part of the original AttrakDiff1 questionnaire; Partial indicates the adjective is similar but not identical to the original pair.

The User Experience Questionnaire (UEQ)

Like the AttrakDiff, the UEQ (User Experience Questionnaire, Laugwitz et al., 2008; see Table 3) assesses pragmatic and hedonic usability with seven-point semantic differential items, using 26 items to measure

Pragmatic usability

  • Perspicuity (4 items)
  • Efficiency (4 items)
  • Dependability (4 items)

Hedonic usability

  • Novelty (4 items)
  • Stimulation (4 items)
  • Attractiveness (6 items)

Response options range from -3 to +3. Seven UEQ items have the same anchors as the AttrakDiff2; four have one anchor in common. The location of the common items is reasonably consistent with AttrakDiff2, with a few exceptions. For example, dull-captivating and unimaginative-creative are part of AttrakDiff2’s Hedonic-Stimulation construct, while creative-dull is an item in UEQ’s Hedonic-Novelty construct. Discouraging-motivating is a pair in AttrakDiff2’s Attractiveness construct; motivating-demotivating is in UEQ’s Hedonic-Stimulation construct.

ConstructItemAnchor 1Anchor 2In AttrakDiff2?
Pragmatic -1not understandableunderstandable
Perspicuity2easy to learndifficult to learn
3complicatedeasyPartial
4clearconfusingPartial
Pragmatic -1unpredictablepredictableYes
Dependability2obstructivesupportive
3securenot secure
4meets expectationdoes not meet expectation
Pragmatic -1fastslow
Efficiency2inefficientefficient
3impracticalpracticalYes
4organizedcluttered
Hedonic - 1creativedullPartial
Novelty2inventiveconventionalYes
3usualleading edge
4conservativeinnovativeYes
Hedonic -1valuableinferior
Stimulation2boringexciting
3not interestinginteresting
4motivatingdemotivatingPartial
Attractiveness1annoyingenjoyable
2goodbadYes
3unlikeablepleasing
4unpleasantpleasantYes
5attractiveunattractivePartial
6friendlyunfriendly

Table 3: UEQ items—Yes in the “Same” column indicates the adjective pair is part of the AttrakDiff2 questionnaire; Partial indicates the adjective is similar but not identical to an AttrakDiff2 pair.

The Components of User Experience (modular evaluation) meCUE questionnaire

The meCUE is a standardized questionnaire based on Thüring and Mahlke’s (2007)’s Components of User Experience model. Its latest version is the meCUE2.0 with 33 items. It’s designed to provide a modular evaluation (that’s the “me”) with each of its components:

Instrumental qualities

  • Perceived usefulness (2 items)
  • Perceived usability (3 items)

Non-instrumental qualities

  • Aesthetics (3 items)
  • Status (3 items)
  • Commitment (3 items)
  • Positive emotions (6 items)
  • Negative emotions (6 items)
  • Intention to use (3 items)
  • Loyalty (3 items)
  • Overall experience (1 item)

The meCUE 2.0 is the longest of the three questionnaires and shares some aspects (in addition to pragmatic usability) with other standardized UX questionnaires (e.g., usefulness with the TAM/UMUX-LITE and loyalty with the SUPR-Q).

The meCUE 2.0 items use a seven-point agreement scale except for the rating of overall experience, which is a 21-point bipolar scale anchored with “as bad” and “as good,” with response options from -5 to +5 in half-point increments. The seven-point scales are fully labeled without numbers using the following response option labels: strongly disagree, disagree, somewhat disagree, neither agree nor disagree, somewhat agree, agree, strongly agree. Table 4 shows the wording of the meCUE2.0 items and their associated components.

ModuleComponentItem
IUsefulnessThe functions of the product are exactly right for my goals.
It is quickly apparent how to use the product.
The operating procedures of the product are simple to understand.
UsabilityThe product is easy to use.
I consider the product extremely useful.
With the help of this product I will achieve my goals.
IIVisual aestheticsThe product is creatively designed.
The design looks attractive.
The product is stylish.
StatusThe product would enhance my standing among my peers.
By using this product, I would be perceived differently.
I would not mind if my friends envied me for this product.
CommitmentI could not live without this product.
This product is like a friend to me.
If I ever lost the product, I would be devastated.
IIIPositive emotionsThe product exhilarates me.
The product relaxes me.
The product makes me feel happy.
The product makes me feel euphoric.
The product calms me.
When using this product, I feel cheerful.
Negative emotionsThe product makes me tired.
The product annoys me.
When using this product I feel exhausted.
The product frustrates me.
The product makes me feel passive.
The product angers me.
IVIntention to useIf I could, I would use the product daily.
I can hardly wait to use the product again.
When using the product, I lose track of time.
Product loyaltyI would not swap this product for any other.
In comparison to this product, no others come close.
I would get exactly this product for myself (again) at any time.
VOverall evaluationHow do you experience the product as a whole?

Table 4: meCUE2.0 questionnaire items, modules, and components

Future of Hedonic Usability

The concept of hedonic usability is reasonably compelling, and the research supporting it seems solid. For practical UX work, however, there are some issues with current approaches to its measurement that stand in the way of wide adoption.

Length, lack of global scores, and absence of norms inhibit adoption

Inhibitors to the use of questionnaires such as AttrakDiff2, UEQ, and meCUE2.0 include

  • Length: In their standard forms, there are from 26 to 33 items in these questionnaires. The short forms that have been developed for AttrakDiff2 and UEQ [pdf] may help their adoption.
  • No global scores: For theoretical reasons, the developers of these questionnaires made the design decision to avoid providing global scores. This can make these instruments difficult to use by practitioners who need to make a global conclusion about product quality.
  • Absence of norms: Except for UEQ, there are no published norms for interpreting scores. The work that the UEQ developers have put into developing interpretive norms may pay off in greater future adoption.

Additional concurrent analysis with the SUS and UMUX-LITE needed

Both the SUS and the UMUX-LITE have well-established norms for measuring their perceived usability. It’s possible that one or more constructs in the class of pragmatic-hedonic questionnaires might not just correlate with SUS or UMUX-LITE but might correspond in magnitude well enough to be able to use their open-source norms. There would also be value in systematically assessing how well this class of questionnaires, along with SUS/UMUX-LITE, accounts for variability in outcome metrics such as ratings of overall experience and likelihood-to-recommend. This would help address the question of what hedonic usability measures capture that pragmatic usability measures don’t.

Summary

Pragmatic usability focuses on the task-oriented nature of an experience, whereas hedonic usability focuses more on the appeal, fun, and originality. The concept of hedonic usability is about 20 years old. Over those 20 years, the measurement of hedonic usability has expanded to multiple constructs and is measured using different questionnaires, most commonly AttrakDiff2, UEQ, and meCUE2.0. Based on the content of their items and structural analyses, all three appear to be viable instruments for UX research; although they are all lengthy (26–33 items), which may inhibit their usage. Only the UEQ has benchmarks for the interpretation of its scores, which is of particular value to practitioners.



Sign-up to receive weekly updates.