Without enough of the right participants agreeing to participate and completing your study, the generalizability of your findings are limited.
Here are five approaches you can use to get the right people to participate in your studies. In many cases you can combine these approaches to achieve higher participation rates.
1. Provide an Incentive… to a Point
Incentives are extrinsic motivators and generally increase response rates to surveys in all modes, including phone-based, mailing, Internet, and panel studies. Monetary incentives tend to increase response rates more than gifts, and prepaid incentives (even as little as $1-2) increase response rates more than lotteries or promised incentives delivered upon survey completion. Unsurprisingly, increasing the amount of the incentive also increases participation rates.
Incentives aren’t a panacea though. There is actually some evidence from social science research suggesting that incentives can actually erode participants’ natural motivation to perform well in a study. For example, adults who initially rated a complex task as interesting, and are then paid for their performance based on how well they do, later rate the same activity as less attractive. Interestingly, when participants weren’t paid, or paid a flat rate regardless of performance, their interest in the task remained consistent over time.
In most cases, incentives get more people to take your study but fortunately don’t seem to have an impact on the quality of the data (positively or negatively). The actual incentive you should offer depends on a number of variables, including the burden on the respondent and how difficult people are to find. Here are 13 things to consider when offering incentives.
2. Appeal to Intrinsic Motivation
Some participants can be motivated to take your study if they feel their responses will make a difference. To attract these participants, be sure to include the benefits or impact participants’ responses will have in the study invitation and welcome screen or script.
A recent meta-analysis[pdf] looked at 40 years of research on intrinsic motivation and found that intrinsic motivation is a major predictor of performance quality. If an individual derives natural enjoyment from whatever task they are completing, it’s very rare for them to perform poorly. In examining the effects of intrinsic motivation and extrinsic incentives on performance, researchers found that for quality of performance, intrinsic motivation is the most important factor; whereas for quantity of performance, extrinsic incentives are most impactful.
Gamification attempts to increase user motivation[pdf] and engagement through the use of game elements (e.g., points, leaderboards, badges) in non-game contexts. The thought is that by mimicking the hedonic or playful aspects of a game, it’s possible to build intrinsic motivation.
While we’ve seen gamification extensively in UX research, it also extends to more traditional survey research. For example, there is some evidence that when market research surveys utilize gamification techniques, participation is improved and motivation increases[pdf].
4. Provide Personalized Feedback
Many people participate in research studies to learn something, whether about themselves or the study being conducted. If you can provide respondents with either immediate or delayed information on their responses, response rates will likely increase.
For example, when respondents received personalized feedback[pdf] about their scores on the Big Five personality inventory, they were more likely to rate the survey as interesting and fun. Like providing incentives, the quality of the data in this study didn’t differ significantly between participants who received personalized feedback and those who didn’t.
5. Share the Findings
When possible and if it makes sense, offer to share the results of the findings with respondents. For example, Consumer Reports regularly surveys its millions of members to answer reliability data about their cars. In return, Consumer Reports provides an aggregated set of responses to participants. Another example is the UXPA salary survey (and most industry surveys). Practitioners answer scores of questions about themselves and their organization; in turn, respondents get a comprehensive report[pdf] on industry practices.
Learn More: UX Measurement Boot Camp
Intensive Training on UX Methods, Metrics and Measurement
|Denver: Aug. 5th-7th, 2020|