Like pictures and pixels on a screen, words are a type of user interface.
Complex language, like complex software, can lead to misunderstanding, so words should communicate effectively while being easy to understand.
The solution, to paraphrase William Zinsser, is to use words that are simple and concise—a guideline that also applies to UX questionnaires.
This brings us to the UMUX-Lite, an increasingly popular UX questionnaire. It’s very short (just two items), correlates significantly and corresponds closely with concurrently collected UX and loyalty metrics, and is structurally related to the Technology Acceptance Model.
Similar to the System Usability Scale (SUS), The UMUX-Lite’s scores range from 0 to 100 after converting its multipoint item ratings to 101-point scales and then averaging the two items.
The standard version of the UMUX-Lite (Figure 1) has two agreement items: “{Product} is easy to use” (assessing Perceived Ease-of-Use) and “{Product}’s capabilities meet my requirements” (assessing Perceived Usefulness).
Figure 1: Example of the standard UMUX-Lite (five-point version; created with MUIQ).
A few months ago, we researched to see whether simplifying the perceived usefulness item of the UMUX-Lite had a significant effect on its scores. Specifically, we compared ratings collected with the original item wording with a somewhat simplified wording, replacing “capabilities” with “functionality” and “requirements” with “needs”:
{Product}’s functionality meets my needs.
We were motivated to start this line of research after we—and our clients—observed the striking difference in the vocabulary level of the two items (“easy to use” versus “capabilities meet my requirements”). When a questionnaire is standardized, though, you can’t just change it. You need evidence that the change doesn’t affect the measurement properties of the questionnaire.
Across three studies, we found no significant difference between the means of the original and alternate UMUX-Lite scores, and no significant differences in the distributions of their response options. This was consistent with what we often find when we investigate the effects of slightly different item wording or formats on rating scale measurements.
As part of our ongoing research to further simplify the wording of this item, we simplified it further, replacing “functionality” with “features”:
{Product}’s features meet my needs.
We thought this would bring the wording of the perceived usefulness item much closer in style and simplicity to the perceived ease-of-use item, but we needed to see whether this greater deviation from the original wording had a stronger effect on the resulting UMUX-Lite scores. We ran a study to find out.
The Study
We included the new item as part of a survey of the UX of auto insurance companies, collected in December 2020 using an online U.S.-based panel (n = 332). The survey was programmed in MUIQ to assign the two versions of the items randomly and not appear with each other.
Results: Comparison of Means
Figure 2 shows the mean UMUX-Lite scores from the study (within-subjects comparison). There was no significant difference between the original and simplified UMUX-Lite scores (t(331) = 1.08, p = .28).
Figure 2: Mean UMUX-Lite scores (with 95% confidence intervals. Current = original wording; Simplified = alternate wording).
The mean difference was −.45, with a 95% confidence interval from −1.28 to .37. The confidence interval contains 0, so it’s plausible that there is no real difference. If there is a real difference, the limits of the confidence interval show that it’s trivial, unlikely to exceed 1.3 on the 0−100-point UMUX-Lite scale.
Results: Comparison of Distribution of Response Options
Figure 3 shows the distributions of response options for the original and simplified versions of the perceived usefulness item. The patterns for the original and the simplified wording were consistent and, for each response option, not significantly different (substantial overlap of 95% confidence intervals).
Figure 3: Distributions of response options for the auto insurance survey with 95% confidence intervals. (Current = original wording; Simplified = alternate wording.)
Results: Scale Reliabilities
Scale reliability for the original wording, measured with coefficient alpha, was .79. Reliability for the simplified version was a slightly higher .82. The reliability of both versions exceeded the typical reliability criterion of > .70 for research metrics, supporting the use of either version.
Summary and Takeaways
The original UMUX-Lite questionnaire contains only two items, measuring (1) perceived ease: “The system is easy to use” and (2) perceived usefulness: “The system’s capabilities meet my requirements.”
Shorter words tend to be easier to read, so we have been exploring simpler versions of the perceived usefulness item. Our earlier research found that simplifying the usefulness item by changing “capabilities” to “functionality” and “requirements” to “needs” had no statistical impact on mean scores nor item distributions.
To determine whether simplifying the usefulness item even more by using “features” instead of “functionality” would affect UMUX-Lite measurements, we conducted a study with 332 U.S.-based participants, who reflected on their experience with auto insurance companies. This further simplification had no statistical impact on mean scores nor item distributions.
These results indicate that UX practitioners can use the usefulness version they wish, but unless there’s a compelling reason to do otherwise, they should prefer the simpler wording.
- The system’s capabilities meet my requirements.
- The system’s functionality meets my needs.
- The system’s features meet my needs.
