You write the best survey questions, pretest them to ensure they make sense.
You use the correct response options and even use psychometrically validated questionnaires in your survey.
You’re ready to launch and wait for the responses to roll in…
It’s no fun planning a party no one comes to. The same can be said about surveys that hardly any one takes.
In the never-ending quest to collect data from customers through branded or unbranded surveys, we inevitably encounter the question: should we offer some type of incentive for people to take the survey?
Here are 13 things to consider when deciding whether an incentive is the way to go.
1. Incentives Help Response Rates
Incentives of some form (money, gift cards, iPads, etc) in general increase response rates to web-based and email surveys. If your survey is on the longer side, incentives can also encourage participants to finish[pdf] it and there’s some evidence they help with the quality of open-ended responses. The good news is that they don’t seem to bias the responses systematically (e.g. satisficing).
2. Money Is Usually Better than Gifts
Monetary incentives generally increase response rates more than gifts. Prepaid incentives help more than promised incentives though it’s difficult to prepay people for web surveys (unlike sending money in the mail with a hard-copy survey). But money doesn’t always help; see #12 below.
3. Paying Can be Painful
Mailing out hundreds of $10 checks isn’t fun (we’ve done it), and isn’t easy with most banks. Mailing physical gifts is even harder because you’ll need to collect valid physical addresses. You can use the next best thing to cash: Amazon gift cards and PayPal. Both Amazon and PayPal make it easy for you to pay in bulk but not everyone has a PayPal account and sometimes you can’t use Amazon (e.g. if you’re another retailer!). There are also bulk payment providers who can distribute cash for you but the fee is often as much as the honorarium.
4. Lotteries and Raffles May Be Cheaper and Easier
Instead of paying everyone, you can have a drawing, lottery, or “raffle.” The nice thing about a lottery is that you can offer a more substantial incentive (e.g. an iPad or $500) instead of a more modest honorarium (e.g. $10). For conducting a lottery or raffle, you’ll need to be sure your participants are aware of their true odds of winning (see below on legality issues) and be sure you have a way of sending the gifts or funds electronically (you don’t want to offer a new iPad if they’re out of stock for months!).
5. Lotteries/Sweepstakes Are Easier but Probably Not as Effective
Lotteries are easier and probably cheaper than paying everyone, but there’s some evidence that they aren’t as effective as paying everyone. This is likely a natural consequence of how people view incentives. Which would you have: guaranteed money (even a small amount) or a small chance to win a modest amount? A professional survey taker puts his thoughts succinctly:
“Sweepstakes are for people who can’t calculate probability properly – scrape a decent budget together and pay people for their time.”
6. Lotteries Have Legal Issues
You’d think it’s simple; you’re giving away things! But it’s not that easy unfortunately. In many U.S. states (and probably most countries) you really have to be careful on how you conduct your sweepstakes or lottery. In fact, you might have to get some legal help to ensure you’re not breaking the law! This article cites at least 30 things you should worry about (including displaying the official rules and how the winner is determined).
7. Consider Other Incentives
You don’t have to give away buckets of cash or iPads as an incentive. If you have a product or service people like (e.g. store credit) or other discounts or coupons (if it’s something you can offer) it may be an effective incentive. Again our candid professional survey taker offers his advice:
“If you’re offering a coupon/gift card, it better be good.”
8. Find the Right Amount
Higher incentives generally increase response rates[pdf] but with a diminishing return. The good news is that while higher incentives improve responses, they don’t seem to negatively affect the nature of the responses or the type of people who respond (you just get more people to respond).
The right amount will depend on your audience (higher for higher-paid professions) and the length of the study. You may need to experiment with different amounts to see what works.
Tip: You can do a soft launch (sending an invitation to a small sample) to see what the response rate is likely to be based on your planned incentive.
9. Budget
You of course can only compensate as much as your budget will allow. Even a nominal compensation of $5 adds up quickly if you need more than 1,000 responses. You will likely need to budget for the cost (and time) to get the money to these scores of participants.
Two things to watch for when paying participants directly:
- Be sure you have a survey quota. Some surveys can fill fast and you don’t want to be on the hook for 2,000 payments if you only budgeted for 1,000.
- If you have to oversample (because of rejecting poor quality responses) you will likely have to decide whether to compensate all or just the responses you thought were worth retaining.
If you’re running a lottery or raffle, you’ll again want to consider the costs of obtaining the incentive and shipping costs if applicable. And if you’re using discount codes, be sure that’s factored into your actual sales margin.
10. More Legal Issues
Not everyone can receive money or gifts. Physicians, radiologists, and government employees are among the many participant types who are often not allowed to receive money at all or from certain businesses. For example, there are rules against paying doctors, even nominal gifts, from pharmaceutical companies due to the potential conflict of interest.
11. Appeal to the Interest of Your Participants
If the participants feel like their voice will make a difference you may not need any incentive. This is especially the case if your survey is generally short (think under 10 minutes). For example, if a customer feels like their opinion will help improve a product or service they use, they’re more likely to respond. Or if you have a passionate group of users, they will likely provide their opinions at much higher rates (e.g. non-profits).
12. Money May Also Hurt Response Rates
Not only does compensating hit the bottom line, in some cases it may negatively affect your response rate. It may seem like offering an incentive is a no-brainer way to get more people to participate. But money is not just like any other type of incentive; people treat it differently.
If your participants are already intrinsically motivated to provide their feedback (e.g. because they believe in your cause or product), offering money can backfire. There’s less research on this effect in survey incentives but it’s a well-documented effect in many contexts.
13. Don’t Coerce
While it’s generally less of a concern for online surveys or applied research, there is often a concern about incentives being coercive, especially for certain disadvantaged populations. In the event you are surveying an economically disadvantaged group, be sure your incentive isn’t coercing people to feel they NEED to respond.
