### You Can Report Percentages with Small Samples, but Should You?

In an earlier article, we demonstrated how it’s completely permissible from a statistical perspective to report numbers when studies have very small sample sizes (fewer than ten people). When you use numbers, you can present them as raw numbers, fractions, or percentages. You can present them in a report, on a train, or on a

### Should You Report Numbers or Percentages in Small-Sample Studies?

“Don’t include numbers when reporting the results of small-sample research studies!” “If you must, definitely don’t use percentages!” “And of course, don’t even think about using statistics!” We regularly hear variations of this advice from well-intentioned researchers, often senior ones. In 2005, we encountered this debate among UX professionals when we participated in a workshop

### How to Compare Two Proportions with the N−1 Two-Proportion Test

Proportional data is common in both UX research and the larger scientific literature. You can use proportions to help make data-driven decisions just about anywhere: Which design converts more? Which product is preferred? Does the new interface have a higher completion rate? What proportion of users had a problem with registering? Consequently, you’ll likely want to

### What a Randomization Test Is and How to Run One in R

The two-sample t-test is one of the most widely used statistical tests, assessing whether mean differences between two samples are statistically significant. It can be used to compare two samples of many UX metrics, such as SUS scores, SEQ scores, and task times. The t-test, like most statistical tests, has certain requirements (assumptions) for its

### 5 Techniques to Identify Clusters In Your Data

Understanding who your users are and what they think about an experience is an essential step for measuring and improving the user experience. Part of understanding your users is understanding how they are similar and different with respect to demographics, psychographics, and behaviors. These groupings are often called clusters or segments to refer to the

### 6 Ways to Visualize Statistical Significance

Researchers rely heavily on sampling. It’s rarely possible, or even makes sense, to measure every single person of a population (all customers, all prospects, all homeowners, etc.). But when you use a sample, the average value you observe (e.g. a completion rate, average satisfaction) differs from the actual population average. Consequently, the differences between designs

### The Difference between Observed and Latent Variables

You can’t see customer satisfaction. You can’t see usability. There isn’t a thermometer that directly measures someone’s intelligence. While we can talk about satisfied customers, usable products, or smart people, there isn’t a direct way to measure these abstract concepts. And clearly these concepts vary. We’ve all had experiences that left us feeling unsatisfied or

### Controlling for Brand Attitudes in UX Studies

There are a number of variables that affect UX metrics. In most cases though, you’ll simply want to measure the user experience and not these other “nuisance variables” that may mask the experience users have with an interface. This is especially the case when making comparisons. In a comparative analysis you use multiple measures to

### 5 Steps for Getting Started with Statistics for Research

Statistics can be daunting, especially for UX professionals who aren’t particularly excited about the idea of using numbers to improve designs. But like any skill that can be learned, it takes some time to understand statistical concepts and put them into practice. Most participants at our UX Boot Camp go from little knowledge of statistics