Do You Need A Random Sample For Your Usability Test?

Usability tests are conducted on samples of users taken from a larger user population. In usability testing it is hard enough to recruit and test users let alone select them randomly from the larger user population. Samples in usability studies are almost always convenience samples. That is, we rely on volunteers to participate in our

Read More »

Will Five Users Really Find 85% Of All Usability Problems?

If you ask five users to take a look at a website or application you will find usability problems. If you fix those problems then ask another five users you will get another set of problems. Over time there will be fewer and fewer problems found, but a new set of users will still continue

Read More »

Why you only need to test with five users (explained)

One question I get a lot is, Do you really only need to test with 5 users? There are a lot of strong opinions about the magic number 5 in usability testing and much has been written about it (e.g. see Lewis 2006PDF). As you can imagine there isn’t a fixed number of users that

Read More »

Deriving a Problem Discovery Sample Size

Nielsen derives his “five users is enough” formula from a paper he and Tom Landauer published in 1993. Before Nielsen and Landauer James Lewis of IBM proposed a very similar problem detection formula in 1982 based on the binomial probability formula.[4] Lewis stated that: The binomial probability theorem can be used to determine the probability

Read More »

Calculating Sample Size for Task Completion (Discrete-Binary Method)

One of the biggest and usually first concerns levied against any statistical measures of usability is that the number of users required to obtain “statistically significant” data is prohibitive. People reason that one cannot with any reasonable level of confidence employ quantitative methods to determining product usability. The reasoning continues something like this: “I have

Read More »

Calculating Sample Size for Task Times (Continuous Method)

We already saw how a manageable sample of users can provide meaningful data for discrete-binary data like task completion. With continuous data like task times, the sample size can be even smaller. The continuous calculation is a bit more complicated and involves somewhat of a Catch-22. Most want to determine the sample size ahead of

Read More »
0
    0
    Your Cart
    Your cart is emptyReturn to Shop
    Scroll to Top