You’ve decided to conduct a survey. Congratulations!

Now it’s time to get into the details. In our experience, one of the most soul-crushing difficulties of running surveys is the process of defining and finding participants.

In this article, we’ll go over some of the logistical details you’ll want to sort out before launching your survey. We’ll answer the following questions:

• How should you define target participants?
• Where do you find participants?
• How should you use online panels?
• How do you get people to participate?
• How should you compensate participants?

How to Define Survey Participants

When defining participants, think in terms of concepts and constructs. Concepts include relatively concrete participant attributes such as demographics. Concepts answer questions about who the participants are, where they are, what they do, and when they do it. In UX and CX research, what people do and their prior experiences with products tend to be more important than standard demographics.

In contrast, constructs (e.g., tech savviness, behavioral intentions) tend to be more abstract and multi-faceted than concepts. Figure 1 breaks concepts and constructs into smaller groups to help better define your participants.

Figure 1: Concepts and constructs for survey research.

Your target participant group is probably not completely homogenous. In our experience, most targeted participants for a survey include a mix of the subgroups shown in Figure 1.

For example, a survey on the attitude of purchasing automobile accessories may target all vehicle owners. But it’s better to sample participants who have already purchased accessories for popular brands like BMW and General Motors.

If you have different subgroups, this may affect the questions you ask, the tasks you present, and the overall sample size. One of the most common subgroups is based on prior experience with a product or domain, as in the auto-accessory example. Additional examples include:

• Existing online grocery customers vs. those interested in trying the service
• Owners of a Linksys router vs. owners of other routers
• Current users of Uber vs. users of Lyft vs. prospective ride-sharing users

Six Variables for Defining Populations and Subgroups

For UX and CX surveys, we recommend considering the following six variables when defining a target population.

1. Product experience: If they have used a specific product or service (e.g., Salesforce, QuickBooks, United Airlines).
2. Domain experience: If they have experience with specific knowledge and products in an industry (e.g., accountants, IT security admins, lawyers).
3. Key tasks: What people attempt to accomplish with a product (e.g., filing their taxes, posting products for sale).
4. Attitudes: Brand attitude, intent to purchase a product.
5. Demographics: Age groups (e.g., older than 65 or teens).
6. Accessibility: People who may need assistive technology (e.g., screen readers) or have conditions that impair their abilities (e.g., vision or hearing impaired).

Where to Find Participants

One of the main advantages of surveys is the ability to collect data from many participants quickly. But this means you’ll need access to a lot of participants. The three best sources for this are online panels, existing customer lists, and website intercepts.

We’ve recently noticed a trend (that we recommend) of organizations building their own panels. They often start with external recruiting efforts and panels, which then turn an online panel into a customer list.

Some Things to Know about Online Panels

Online panels are the go-to method for collecting data quickly for online surveys, but they have some drawbacks. Here are nine considerations for their use:

1. Participants belong to multiple panels. People do this to earn more rewards.
2. Multiple panel memberships affect attitudes. Repeated exposure to surveys can lead to inattentive respondents.
3. The length of panel membership also has an effect. This is another way in which repeated exposure can lead to inattentive responses.
4. The average online study lasts less than 20 minutes. Avoid studies that are too long. Study length is a major contributor to dropout.
5. Population estimates can vary. Point estimates obtained from different panels often vary by 15–18 percentage points.
6. UX metrics can vary, too, but not as much as expected. Point estimates for common UX benchmark metrics vary between panels, on average, between 3–10%.
7. Changing panels changes estimates. Try to avoid changing panels when you will make comparisons over time.
8. Probability panels are probably better. Probability panels sample proportionally from the general population. When this is an important research consideration, you should consider such a panel, but they usually come at a higher price.
9. Be wary of panel quality. Panels can vary in quality from each other and over time, so consider starting the study with participants from multiple panels to fill samples and corroborate findings.

Some Good Ways to Get People to Participate

A lot of work goes into planning a survey (writing questions, selecting response options), and it can be frustrating if not enough people take the survey. It’s like planning a party, and no one comes!

Without enough of the right participants agreeing to participate and completing your survey, the generalizability of your findings is limited. Here are five approaches for getting the right people to participate, which you can often combine to achieve higher participation rates.

1. Provide an incentive … to a point. Incentives get more people to take surveys, but they don’t seem to affect the quality of responses.
2. Appeal to intrinsic motivation. When participants feel their responses will make a difference, they will be more motivated to take the survey, and motivated participants provide higher-quality data.
3. Use game elements. Participants who can earn points, leaderboards, or badges are more engaged.
4. Provide personalized feedback. Providing personalized feedback about someone’s responses can increase engagement.
5. Share the findings. When possible, offer to share the survey findings with participants.

What to Consider When Compensating Participants

In the never-ending quest to find participants for survey studies (and any research, for that matter), you’ll likely confront the question of whether you should offer—and how much—to pay participants. Here are 13 things to consider when deciding whether an incentive is the way to go for your survey.

1. Incentives help response rates. And they do so without seeming to systematically bias responses.
2. Money is usually better than gifts. Cash is king.
3. Paying can be painful. Amazon gift cards and PayPal rewards are easier than mailing hundreds of checks, but they won’t work for every participant.
4. Lotteries may be cheaper and easier. But it is important to execute them properly.
5. Although they are easier, lotteries are probably not as effective. This is most likely due to people’s beliefs about their odds of winning.
6. Lotteries also have legal issues. Do not conduct a lottery without getting guidance to ensure you are not putting your organization in legal jeopardy.
7. Consider other incentives. In some situations, you can offer store credit, discounts, or coupons, but if you do, they need to appeal to the potential participants.
8. Find the right amount. This depends on your audience and the length of the survey.
9. Work within your budget. And be careful you don’t accidentally collect more responses than planned.
10. Know the potential legal issues. People working in certain occupations (e.g., physicians, government employees) may be legally prohibited from receiving payment for participation.
11. Appeal to the interest of your participants. When participants care about your survey research, they may be motivated enough to participate without compensation (especially if the survey is short).
12. Money can sometimes hurt response rates. This is uncommon, but some participants might be offended when offered a monetary incentive.
13. Don’t coerce. There can be an ethical concern when offering payment to economically disadvantaged groups. You don’t want people to feel they must participate even if they don’t really want to.

Looking to Learn More?

Join our upcoming webinar on Surveying the User Experience.

We’ve written a lot more on defining and finding participants and the entire survey creation process, from writing questions, selecting response options, and collecting and analyzing data. You can find plenty more in our book, Surveying the User Experience. We also have a companion course that follows the book on MeasuringUniversity.com.  Consider using our MUiQ platform to build your own surveys and recruit participants.