Since its development in the 1980s, System Usability Scale (SUS) score analysis has focused on estimating and comparing means. The mean is one type of average. Would it be more appropriate to estimate and compare SUS medians instead of means? To investigate this question, we analyzed and compared SUS means and medians collected from over
In an earlier article, we demonstrated how it’s completely permissible from a statistical perspective to report numbers when studies have very small sample sizes (fewer than ten people). When you use numbers, you can present them as raw numbers, fractions, or percentages. You can present them in a report, on a train, or on a
“Don’t include numbers when reporting the results of small-sample research studies!” “If you must, definitely don’t use percentages!” “And of course, don’t even think about using statistics!” We regularly hear variations of this advice from well-intentioned researchers, often senior ones. In 2005, we encountered this debate among UX professionals when we participated in a workshop
Proportional data is common in both UX research and the larger scientific literature. You can use proportions to help make data-driven decisions just about anywhere: Which design converts more? Which product is preferred? Does the new interface have a higher completion rate? What proportion of users had a problem with registering? Consequently, you’ll likely want to
The two-sample t-test is one of the most widely used statistical tests, assessing whether mean differences between two samples are statistically significant. It can be used to compare two samples of many UX metrics, such as SUS scores, SEQ scores, and task times. The t-test, like most statistical tests, has certain requirements (assumptions) for its
Understanding who your users are and what they think about an experience is an essential step for measuring and improving the user experience. Part of understanding your users is understanding how they are similar and different with respect to demographics, psychographics, and behaviors. These groupings are often called clusters or segments to refer to the
Researchers rely heavily on sampling. It’s rarely possible, or even makes sense, to measure every single person of a population (all customers, all prospects, all homeowners, etc.). But when you use a sample, the average value you observe (e.g. a completion rate, average satisfaction) differs from the actual population average. Consequently, the differences between designs
You can’t see customer satisfaction. You can’t see usability. There isn’t a thermometer that directly measures someone’s intelligence. While we can talk about satisfied customers, usable products, or smart people, there isn’t a direct way to measure these abstract concepts. And clearly these concepts vary. We’ve all had experiences that left us feeling unsatisfied or
There are a number of variables that affect UX metrics. In most cases though, you’ll simply want to measure the user experience and not these other “nuisance variables” that may mask the experience users have with an interface. This is especially the case when making comparisons. In a comparative analysis you use multiple measures to
Statistics can be daunting, especially for UX professionals who aren’t particularly excited about the idea of using numbers to improve designs. But like any skill that can be learned, it takes some time to understand statistical concepts and put them into practice. Most participants at our UX Boot Camp go from little knowledge of statistics
