Unless you plan to survey every member of your target population, you’ll need to work with a sample. But even in cases where you can survey everyone, you might not want to because of survey fatigue and costs. What’s more, most populations are fluid. There may be only a hundred users of a financial product
The System Usability Scale (SUS) is one of the oldest standardized UX questionnaires. John Brooke is now retired; should the questionnaire he developed almost 40 years ago accompany him on the beach with a piña colada? After all, the SUS was developed when there were green-screen computer monitors. How can it possibly apply to mobile
All of us at MeasuringU® wish you a Happy New Year! In 2023, we posted 49 articles and continued to add features to our MUiQ® UX testing platform to make it even easier to develop studies and analyze results. We hosted our eleventh UX Measurement Bootcamp, again as a blended virtual event with a combination
The standard deviation is the most common way of measuring variability or “dispersion” in data. The more the data is dispersed, the more measures such as the mean will fluctuate from sample to sample. That means higher variability (higher standard deviations) requires larger sample sizes. But exactly how much do standard deviations—whether large or small—impact
Are customers more satisfied this quarter than last quarter? Do users trust the brand less this year than last year? Did the product changes result in more customers renewing their subscriptions? When UX researchers want to measure attitudes and intentions, they often ask respondents to complete multipoint rating scale items, which are then compared with
“Don’t include numbers when reporting the results of small-sample research studies!” “If you must, definitely don’t use percentages!” “And of course, don’t even think about using statistics!” We regularly hear variations of this advice from well-intentioned researchers, often senior ones. In 2005, we encountered this debate among UX professionals when we participated in a workshop
Sample size estimation is an important part of study planning. If the sample size is too small, the study will be underpowered, meaning it will be incapable of detecting sufficiently small differences as statistically significant. If the sample size is too large, the study will be inefficient and cost more than necessary. A critical component
Is product satisfaction above average? Is it best in class? Do customers have a favorable or unfavorable opinion of the current product? When UX researchers want to measure attitudes and intentions, they often ask respondents to complete multipoint items like the one shown in Figure 1. It’s also common to set a target benchmark for
Sample size computations can seem like an art. Some assumptions are involved when computing sample sizes, but it should be more math than magic. A key ingredient needed to cook up a sample size estimate is the standard deviation. You need yeast to make bread, and you need a measure of variability to make an
What is the impact if you sample a lot of your population in a survey? Many statistical calculations—for example, confidence intervals, statistical comparisons (e.g., the two-sample t-test), and their sample size estimates—assume that your sample is a tiny fraction of your population. But what if you have a relatively modest population size (e.g., IT decision-makers
