If you ask five users to take a look at a website or application you will find usability problems. If you fix those problems then ask another five users you will get another set of problems. Over time there will be fewer and fewer problems found, but a new set of users will still continue
While testing with five users might reveal 85% of problems that impact 31% of users (given a set of tasks and user-type), it doesn’t mean you’re finding 85% of the critical problems. Are severe usability problems likely to occur more frequently, less frequently or is problem severity independent of frequency? The data on this is
Let’s imagine you are testing five users as part of an iterative testing services to find and fix problems. During the test only one user encounters a problem with logging in. To fix this particular problem would take a lot of effort and the small sample size is met with skepticism from the overburdened and
Why isn’t usability testing done more? And when it is done why is the sample size small? One major reason is the cost. It takes a lot of money and time to bring users into a lab and conduct a usability test. Even if users don’t get compensated for their time, it still takes a
