Best Practices for Using Statistics on Small Sample Sizes

Some people think that if you have a small sample size you can’t use statistics. Put simply, this is wrong, but it’s a common misconception. There are appropriate statistical methods to deal with small sample sizes. Although one researcher’s “small” is another’s large, when I refer to small sample sizes I mean studies that have

Read More »

Five Critical Quantitative UX Concepts

If you’re in User Experience, chances are you probably didn’t get into the field because of your love of math. As UX continues to mature it’s becoming harder to avoid using statistics to quantify design improvements. One of my goals is to help make challenging concepts more approachable and accessible. Last week Jim Lewis and

Read More »

How to Find the Right Sample Size for A Usability Test

It’s usually the first and most difficult question to answer when planning a usability evaluation: What sample size do I need? There are some who will just say it doesn’t matter what the sample size is because usability is qualitative…and after all any users are better than none. Others will say that you only need

Read More »

How Many Customers Should You Observe?

Timing, luck and perseverance all play a role in making a successful product. But so does observing and understanding your customers’ problems. The number of customers you need to observe will depend on how common customer behaviors are and how certain you need to be. Building a successful product means building something that customers want

Read More »

How Many Users Do People Actually Test?

If you’re familiar with usability testing then you’re familiar with the magic number 5. Five users will on average find most of the problems that affect at least one-third or more of your users. If problems are less common, then you will need to test more users to find and fix them. On many high-traffic

Read More »

A Brief History Of The Magic Number 5 In Usability Testing

Wondering about the origins of the sample size controversy in the usability profession?  Here is an annotated timeline of the major events and papers which continue to shape this topic. The Pre-Cambrian Era (Up to 1982) It’s the dawn of Usability Evaluation and the first indications of diminishing returns in problem discovery are emerging. 1981:

Read More »

What is a Representative Sample Size for a Survey?

Will users purchase an upgrade? What features are most desired?  Will they recommend the product to a friend? Part of measuring the user experience involves directly asking users what they think via surveys. The Web has made surveys easy to administer and deliver. It hasn’t made the question of how many people you need to

Read More »

What Five Users Can Tell You That 5000 Cannot

With usability testing it used to be that we had to make our best guess as how users actually interacted with software outside a contrived lab-setting. We didn’t have all the information we needed. Knowing what users did was in a sense a puzzle with a lot of missing pieces. Web-analytics provides us with a

Read More »

Do You Need A Random Sample For Your Usability Test?

Usability tests are conducted on samples of users taken from a larger user population. In usability testing it is hard enough to recruit and test users let alone select them randomly from the larger user population. Samples in usability studies are almost always convenience samples. That is, we rely on volunteers to participate in our

Read More »

Will Five Users Really Find 85% Of All Usability Problems?

If you ask five users to take a look at a website or application you will find usability problems. If you fix those problems then ask another five users you will get another set of problems. Over time there will be fewer and fewer problems found, but a new set of users will still continue

Read More »
0
    0
    Your Cart
    Your cart is emptyReturn to Shop
    Scroll to Top