While UX research may be a priority for you, it probably isn’t for your participants. And participants are a pretty important ingredient in usability testing.
If people were predictable, reliable, and always did what they said, few of us would make a living in improving the user experience!
Unfortunately, people don’t always show up when they say they will for your usability test, in-depth interview, or other research study. They get busy, forget, or have other things they need to do other than participate in your study.
For in person and remote studies, always plan for a specific percentage of participants not showing up. The “typical” no-show rate we see for moderated (in person or remote) studies is between 10% and 20%. But we’ve seen as many as half the people not showing.
We’ve been conducting studies every week at MeasuringU for years and have implemented several steps to get no-show rates down to 0. Here are eight of them.
1. Establish both phone and email contact.
Don’t just email participants for scheduling; speak to them on the phone and let them know there are people waiting for them. A simple conversation lets participants know you’re real and counting on them.
2. Communicate it’s a dedicated slot.
Despite the popularity of usability testing and other one-on-one research, most people will have never participated in anything like it. The closest thing people are likely to be familiar with is a focus group, where they participate with other people. There may be a perception that the “show will go on without me” if people decide not to attend. Let participants know it’s a one-on-one study reserved just for them; remind them to let you know if they can’t attend or need to reschedule.
3. Make reminder calls.
One day prior to the study, follow up with a call to speak with the participant (or leave a message). If you’re unable to get ahold of them, ask them to send an email, return the call, or text to confirm their appointment.
4. Include clear directions.
It may seem obvious, but if people can’t find your facility or where to park, they may just give up and go home. Or if they do make it, trouble finding parking or your entrance could delay them significantly and probably put them in a bad mood for a study!
5. Ensure they know what to expect.
In your email and phone communications with participants, let them know what will be expected of them (e.g., asked to work on accounting software), their time commitment (e.g., 1 hour), and honorarium and how it will be paid (e.g., $100 cash or Visa gift card).
6. Exploit the consistency bias.
In our screeners, we ask participants to rate themselves on a number of items that tap into the ideas of personal reliability and accountability. For example, participants are asked to agree or disagree to statements such as “People can count on me to be on time.” and “I follow through on the commitments that I make.”
We don’t expect people to rate themselves low; that’s not the point. We’re taking advantage of the consistency principle here. People are more likely to follow through on future actions to be consistent with prior commitments. Rating yourself as reliable and someone who goes to appointments would be inconsistent with someone who doesn’t call or doesn’t show up. This doesn’t entirely eliminate no shows but we suspect it helps (a future analysis can further investigate its efficacy).
7. Over-recruit.
Even with our best efforts, things happen, and people cancel (often no call, no show); even people sourced from professional recruiting firms or our clients. We’ve found the best thing to do is anticipate a 10% to 20% no-show rate and over-recruit. If you need 20 participants, plan, budget, and recruit 22 to 24. If everyone makes it, you have a few extra data-points.
8. Over-schedule.
Over-recruiting is often not enough to ensure your study stays on schedule. You may have a couple extra participants ready to participate, but a no call, no show means you need to scramble quickly to make up the lost person and time. Having people “float” and always available can be both costly and difficult. Instead, when we need to stick to a tight schedule and ensure a minimum number of participants, we schedule extra participants ahead of time.
This gives us more options. If all the participants show up as planned, you can use the extra participants or cancel them and pay the participants anyway for blocking their time. Of course, if too many participants cancel, budget for a bottle of wine for your recruiter and facilitators!
