Sample Size Calculator for Discovering Problems in a User Interface
This calculator produces an estimate of the number of users needed to discover the specified percent of total problems. It uses the Good-Turing and Normalization procedure.
Sample Size from an estimate of Problem Occurrence (p)If the probability of detecting a UI problem is known in advance, use this portion of the calculator to estimate the total number of users needed to uncover on average the specified percentage of problems (e.g. 90%). The calculator is based on the binomial probability formula.
Estimate Problem Occurrence (p) then Sample Size
This portion of the calculator first builds an estimate of the probability of detecting a UI problem (from sample data). It then produces an estimate of the number of users needed to discover the specified percent of total problems. It uses the Good-Turing and Normalization procedure as outlined by Lewis (2001) and further discussed in (Turner, Lewis & Nielsen 2006).
Lewis, James (2001) "Evaluation of Procedures for Adjusting Problm-Discovery Rates Estimated from Small Samples" in The International Journal of Human-Computer Interaction 13(4) p. 445-479
Turner, C. W., Lewis, J. R., and Nielsen, J. (2006). Determining usability test sample size. In W. Karwowski (Ed.), International Encyclopedia of Ergonomics and Human Factors (pp. 3084-3088). Boca Raton, FL: CRC Press.