{"id":318,"date":"2016-08-23T22:30:39","date_gmt":"2016-08-23T22:30:39","guid":{"rendered":"http:\/\/measuringu.com\/samplesize-start\/"},"modified":"2021-01-28T06:30:15","modified_gmt":"2021-01-28T06:30:15","slug":"samplesize-start","status":"publish","type":"post","link":"https:\/\/measuringu.com\/samplesize-start\/","title":{"rendered":"Getting Started Finding the Right Sample Size"},"content":{"rendered":"
<\/a>Most methods in UX<\/a> rely on collecting data (behavioral and attitudinal<\/a>) from a sample of participants. <\/p>\n But knowing how many participants you should use is not a simple question. <\/p>\n What’s particularly difficult about learning how to compute the right sample size for a study is that books and articles can get overly technical; it’s hard to know whether the advice is relevant to an applied research setting. In general, it can be hard to know where to start. <\/p>\n Here are five steps to help you start computing the right sample size for your study.<\/p>\n The sample size you use for your research does matter. You may hear people dismiss the whole question of what sample size you need as irrelevant. It is very relevant. This is an unfortunate consequence where I believe people get confusing and conflicting advice. <\/p>\n When you can’t measure all your users, you have to deal with the very real consequences of sampling error<\/a>. The number of participants you test has an impact on how generalizable your findings are. This applies to both qualitative<\/a> and quantitative research<\/a> as well as generative and evaluative research.<\/p>\n There’s not a one-size-fits-all answer. The sample size you need is not always 5, 30, 100, or 1,000. <\/p>\n Instead, the sample size you need depends on the goals and type of study. In general, most UX research falls into one of three study types, each with a different sample size computation.<\/p>\n The representativeness of a sample<\/a> of participants is different than the size of the sample. Having responses from a lot of the wrong people doesn’t make a lot of sense, so you want it to represent the population from which you are making inferences. This is representativeness. <\/p>\n This concept often gets confused with the sample size. People will often ask whether a sample size of 20, 30, or some amount is representative. Knowing the size alone is not enough to answer that question. For a generally homogenous group of customers, a sample size of 2 can be representative, as can a sample size of 200. The difference is in the level of precision. Higher sample sizes afford you with more precision about the information you obtain. <\/p>\n When making comparisons, the size of the difference matters a lot<\/a>. Smaller differences require larger sample sizes. Large differences between whatever you are comparing means a smaller sample size will suffice. When you want to detect differences, such as completion rates, two groups in a survey, or times to register on an old versus new app, you first need to think if such a difference does exist, whether it’s likely to be small or large. <\/p>\n This concept also applies to problem discovery studies. When discovering problems, the more users a problem impacts, the smaller the sample size needed to uncover these problems. Conversely, problems that affect a small number of users are very hard to detect in small sample studies and require larger sample sizes. The same applies to observing behaviors in a contextual inquiry<\/a>.<\/p>\n Understanding a concept and putting it to use helps make it understandable and actionable. 1. The Sample Size Matters<\/h2>\n
2. One Size Does NOT Fit All<\/h2>\n
\n
3. Representativeness Is Different than Sample Size<\/h2>\n
4. The Magnitude of the Effect Often Matters Most <\/h2>\n
5. Practice Finding the Sample Size for Common UX Research Studies (And the Theory Too)<\/h2>\n
To give you some practice, here are eight examples<\/a> for finding the right sample size in a formative usability test, contextual inquiry, surveys, benchmarks, and comparative studies. <\/p>\n