User and customer research fundamentally rely on collecting data from users and customers.
But it can be a constant challenge to find the right number and type of qualified participants .
Even when you find the right participants, there’s a limit to how much time they’re willing to spend filling out a survey.
A common question we get from clients is how long is too long for an online study. It’s a good question because one of the major causes of online study dropout is simply the length of a study.
In general, the longer the study, the more people drop out, even those who are paid. There’s a balance between packing a study with as many questions as possible while also keeping the study short.
The answer to the question—how long is the typical study?—like many other answers in online research is “it depends,” which is hardly helpful. So we looked for data to provide some guidance.
A Study on Study Times
Online studies vary from simple surveys that have only one or two questions to complex choice-based conjoint questions to task-based unmoderated usability studies.
To understand how long a “typical” online study takes, we started with our data. We conduct dozens of online studies a year and compiled data from the last three years. We removed moderated studies, which left us with data from 260 studies.
The distribution of median times is shown in Figure 1.
Figure 1: Distribution of median times across 260 online studies conducted by MeasuringU.
The median time for the 260 studies is 9 minutes. Half the study times (called the interquartile range) fall between 5 and 17 minutes. That is, 75% of studies took less than 17 minutes and only 25% were less than 5 minutes.
Keep in mind these are median times, meaning half the respondents in each study took longer than the value shown. So a study with a median time of 23 minutes had some participants taking longer than 30 minutes to complete it.
While there is a large variation in the question types and goals of each study, we categorized our studies into two types: surveys and unmoderated usability studies. In unmoderated usability studies, participants were asked to complete tasks on websites and answer questions about the experience.
We found that for the 130 studies classified as surveys, the median time was 5 minutes, with the 25th percentile at 3 minutes and the 75th percentile at 9 minutes. The 130 unmoderated studies took more than twice longer with a median time of 14 minutes, with the 25th percentile at 9 minutes and the 75th percentile at 24 minutes.
Figure 2 shows the box and whisker plots for the median and interquartile range for both types of studies.
Figure 2: Box and whisker plots showing the median and interquartile range of online study times for 130 surveys and 130 unmoderated usability tests.
Larger Industry Data
This analysis is the only one I know of that focuses on UX online research. Of course, we don’t know how representative our data is relative to all online studies; please keep this in mind as you compare your study times to ours (even if you are in the UX industry).
To help corroborate our data, we looked to the literature. I found a rare and interesting analysis of online surveys from 11 online panel companies from 2012[pdf]. The study methodology provides insight into hundreds of surveys from multiple organizations using multiple panels.
| Panel | Time |
| 1 | 22.1 |
| 2 | 20.2 |
| 3 | 17.3 |
| 4 | 17.7 |
| 5 | 17.5 |
| 6 | 16.1 |
| 7 | 10.7 |
| 8 | 21.2 |
| 9 | 18.3 |
| 10 | 9.6 |
| 11 | 19.6 |
| Mean | 17.3 |
Table 1: Average online survey length from 11 panels.
Across the 11 panels the mean study time was 17.3 minutes, with one panel having a low of 10 minutes and another with a high of 22 minutes. The authors noted that some surveys lasted 80 minutes and that 30-to-40 minute surveys were not rare. In fact, 48% of studies lasted 20 minutes or more, including 13% that were 30 minutes or more.
Discussion
The panel meta-analysis suggests our studies may be on the short side. This could be due to a number of factors. We tend to use a mix of volunteers (usually from client customer lists), participants from website intercepts, and paid panel participants. The meta-analysis authors suggest that organizations can request more of participants’ time when they’re paid (and clearly some organizations and panel companies push this idea to the limit). However, both data sets showed studies that last more than 30 minutes, while less common, still make up between 9% (MeasuringU’s data) and 13% (panel meta-analysis).
Another reason our studies may be shorter than average is that we are concerned about sample quality from high dropout rates and fatigue (even from paid participants). Consequently, we tend to prefer multiple short studies to a long one.
Summary
This analysis of online study times found:
- The median study time across MeasuringU’s 260 online surveys and unmoderated studies was 9 minutes with 75% taking less than 17 minutes.
- Unmoderated usability studies took a median of 14 minutes compared to 5 minutes for surveys.
- A meta-analysis on hundreds of online surveys found an average time of 17 minutes.
- Paid online panel research may have longer study times as participants are expected to spend more time for their paid compensation.
- Across both sources of data these numbers suggest a “typical” study time is less than 17 minutes and a “long” study is one lasting 30 minutes or longer.
