It seems like every few years a new standardized UX measure comes along. Standardization of UX measurement is a good thing for researchers and practitioners. Having common methods and definitions helps with objectivity, generalization, economy, and professional communication.
At MeasuringU, we pay a lot of attention to the continuing evolution of standardized UX measurement. The UMUX-Lite is a relatively recent measure that has been attracting attention among UX practitioners. There are several reasons for this attention:
- It’s an ultra-short questionnaire (just two items).
- It correlates significantly with concurrently collected UX and loyalty measures.
- Its means tend to correspond closely with concurrently collected System Usability Scale (SUS) means, enabling their interpretation with published SUS norms.
- Regression analyses have shown that the UMUX-Lite items are related to the Perceived Ease-of-Use and Perceived Usefulness components of the Technology Acceptance Model (TAM).
In its standard form (Figure 1), the UMUX-Lite has two agreement items: “{Product} is easy to use” (assessing Perceived Ease-of-Use) and “{Product}’s capabilities meet my requirements” (assessing Perceived Usefulness).
Like the SUS, its scores range from 0 to 100, using standard methods to convert its multipoint item ratings to 101-point scales and then averaging the two items.
Figure 1: Example of the standard UMUX-Lite (five-point version, created with MUIQ).
The UMUX-Lite Usefulness Item
The UMUX-Lite inherited its items from an earlier questionnaire, the Usability Metric for User Experience (UMUX). The wording of the first item is clear. We have successfully used the second item in its current form, but we often wondered whether it was possible to simplify its wording without affecting the properties of the UMUX-Lite’s mean or distributions of response options. Our clients have also asked whether we could simplify the wording.
A similar alteration affected Item 8 in the System Usability Scale (SUS). It was originally “I found the system very cumbersome to use” but some participants had trouble understanding what “cumbersome” meant, so it has evolved over the last 30 years into “awkward.”
In this article, we describe our first attempt to simplify the wording of the UMUX-Lite.
Investigating the Wording of the UMUX-Lite Usefulness Item
We developed an alternate Usefulness item that replaced “capabilities” with “functionality” and “requirements” with “needs” (see Figure 2). In the spirit of standardization, we included this item in three recent studies so we could see how well it worked when used in place of the original item.
Figure 2: Example of a modified UMUX-Lite (five-point version, created with MUIQ).
These are the three studies that used the altered wording:
- Entertainment (n = 328): Retrospective UX survey of five video entertainment services
- Meeting Software (n = 198): Retrospective UX survey of four video meeting services
- Online Purchase (n = 348): Unmoderated usability study of four shopping websites
Results
Figure 3 shows the mean UMUX-Lite scores from the three studies (all within-subjects comparisons). In all three studies, there was no significant difference between the original and alternate UMUX-Lite scores.
Figure 3: Mean UMUX-Lite scores (with 95% confidence intervals) from three studies (LiteReqs = computed with original wording; LiteNeeds = computed using alternate wording).
Table 1 summarizes the means and mean differences from the studies, including 95% confidence intervals. Across the studies, the confidence intervals around the mean differences all contain 0, meaning that it’s plausible that there is no real difference. The upper and lower limits of the confidence intervals indicate that if there are real differences, they’re very unlikely to exceed 1 on the 0–100-point UMUX-Lite scale.
| Study | LiteReqs | LiteNeeds | |d| | 95% CI(d) |
|---|---|---|---|---|
| Entertainment | 77.9 ± 2.0 | 77.6 ± 2.0 | 0.3 | −0.6 to 1.2 |
| Meeting Software | 75.7 ± 2.1 | 75.8 ± 2.1 | 0.1 | −0.7 to 0.9 |
| Purchase | 89.5 ± 1.5 | 89.4 ± 1.6 | 0.1 | −0.5 to 0.8 |
Table 1: Summary of mean differences for UMUX-Lite scores from three studies (LiteReqs = computed with original wording; LiteNeeds = computed using alternate wording; |d| = absolute value of the mean difference; 95% CI(d) = 95% confidence interval around the mean difference).
Figures 4, 5, and 6 show the distributions of response options for the original and alternate versions of the Usefulness item for each study. In each study, the patterns for the original and alternate wording were consistent, and for each response option, not significantly different (substantial overlap of 95% confidence intervals).
Figure 4: Distributions of response options for the Entertainment study with 95% confidence intervals (Reqs = original item wording; Needs = alternate item wording).
Figure 5: Distributions of response options for the Meeting Software study with 95% confidence intervals (Reqs = original item wording; Needs = alternate item wording).
Figure 6: Distributions of response options for the Online Purchase study with 95% confidence intervals (Reqs = original item wording; Needs = alternate item wording).
Summary and Takeaways
The UMUX-Lite, a popular questionnaire among UX professionals, includes only two items. The original wording for the Perceived Ease-of-Use item is simple and clear, but the original wording for the Perceived Usefulness item is less so.
In the research reported in this article, we have compared means and response option distributions for two versions of the questionnaire’s Perceived Usefulness item, finding no practical or statistically significant differences between scores computed with the original and the alternate wordings of the item.
The results are not surprising: the changes we made were small. Probably the most consequential simplification was the replacement of “requirements” with “needs.” These results mean that practitioners should use whichever version they please in their UX research.
As we continue this line of research, we plan to study additional alternate Usefulness items that have simpler words in place of “capabilities” or “functionality.” Stay tuned for those results.
