Sample Size Recommendations for Benchmark Studies

One of the primary goals of measuring the user experience is to see whether design efforts actually make a quantifiable difference over time. A regular benchmark study is a great way to institutionalize the idea of quantifiable differences. Benchmarks are most effective when done at regular intervals (e.g., quarterly or yearly) or after significant design

Read More »

Sample Size in Usability Studies: How Well Does the Math Match Reality?

We’ve written extensively about how to determine the right sample size for UX studies. There isn’t one sample size that will work for all studies. The optimal sample size is based on the type of study, which can be classified into three groups: Comparison studies: Comparing metrics for statistical differences Standalone studies: Estimates a population

Read More »

Getting Started Finding the Right Sample Size

Most methods in UX rely on collecting data (behavioral and attitudinal) from a sample of participants. But knowing how many participants you should use is not a simple question. What’s particularly difficult about learning how to compute the right sample size for a study is that books and articles can get overly technical; it’s hard

Read More »

How to Find the Sample Size for 8 Common Research Designs

What sample size do we need? It’s consistently among the most common questions I get from researchers. It can be a confusing process, but that’s why we cover sample-size planning at the Denver UX Boot Camp. Determining the right sample size for a project is a science–an imprecise science. It’s like appraising a house: you

Read More »

Is Observing One User Worse Than Observing None?

Seeing is believing. Observing just a handful of users interact with a product can be more influential than reading pages of a professionally done report or polished presentation. But what if a stakeholder only has time to watch two or just one of the users in a usability study? Are there circumstances where watching some

Read More »

5 Reasons You Should and Should Not Test With 5 Users

There are a lot of misconceptions about when it is and when it is not appropriate to test with five users. There’s no reason to take an extreme position on this issue and think it’s never acceptable or always the right number. Instead you should understand what you can and cannot learn from just a

Read More »

Best Practices for Using Statistics on Small Sample Sizes

Some people think that if you have a small sample size you can’t use statistics. Put simply, this is wrong, but it’s a common misconception. There are appropriate statistical methods to deal with small sample sizes. Although one researcher’s “small” is another’s large, when I refer to small sample sizes I mean studies that have

Read More »

Five Critical Quantitative UX Concepts

If you’re in User Experience, chances are you probably didn’t get into the field because of your love of math. As UX continues to mature it’s becoming harder to avoid using statistics to quantify design improvements. One of my goals is to help make challenging concepts more servicesable and accessible. Last week Jim Lewis and

Read More »

How to Find the Right Sample Size for A Usability Test

It’s usually the first and most difficult question to answer when planning a usability evaluation: What sample size do I need? There are some who will just say it doesn’t matter what the sample size is because usability is qualitative…and after all any users are better than none. Others will say that you only need

Read More »

How Many Customers Should You Observe?

Timing, luck and perseverance all play a role in making a successful product. But so does observing and understanding your customers’ problems. The number of customers you need to observe will depend on how common customer behaviors are and how certain you need to be. Building a successful product means building something that customers want

Read More »
0
    0
    Your Cart
    Your cart is emptyReturn to Shop
    Scroll to Top