{"id":18,"date":"2010-02-01T00:00:00","date_gmt":"2010-02-01T00:00:00","guid":{"rendered":"http:\/\/measuringu.com\/binomial-probability\/"},"modified":"2021-01-28T06:29:24","modified_gmt":"2021-01-28T06:29:24","slug":"binomial-probability","status":"publish","type":"post","link":"https:\/\/measuringu.com\/binomial-probability\/","title":{"rendered":"If 1 Of 5 Users Has A Problem In A Usability Test Will It Impact 1% Or 20% Of All Users?"},"content":{"rendered":"

\"\"<\/a>Let’s imagine you are testing five users as part of an iterative testing services to find and fix problems. During the test only one user encounters a problem with logging in. To fix this particular problem would take a lot of effort and the small sample size is met with skepticism from the overburdened and overcommitted development team. They say “We really don’t know whether this problem will affect 1 out of 5 users (20%) or 1 out of 100 users (1%) and don’t have the resources to fix all edge cases.” They send you packing to ponder the merits of larger sample sizes or changing careers.<\/p>\n

While many people have come to accept testing with small sample sizes in usability testing, there is still a lot of discomfort when extrapolating the results to the entire user population.\u00a0 It turns out the developers are right: we don’t know whether it will affect 1%, 20% or another percentage. But consider these uncertainties:<\/p>\n