The real value is not in the equipment or technology but in the technique.
Usability testing is not a focus group. Nor is usability testing a product demo. You shouldn’t lead participants through a product as if it were a demo and ask them if they “like” something. Uncovering problems users encounter with an interface and fixing them early and often should lead to a better (and often more profitable) experience.
If you don’t have a dedicated usability lab and you want to do more than grab unsuspecting people in Starbucks to try out your designs, you’ll want to put some thought and effort into a usability testing setup. You can run a test on a shoestring budget ($100) or more than $3k/day.
We’ll assume you have something to test, a way to find participants, some sort of compensation (people still like free t-shirts!), and a way to plan for no-shows. You’ll need a space and some technology, both of which can get costly and complicated, but don’t have to.
At a minimum, you’ll need a quiet place to have one-on-one sessions with participants. This can be any room you find, or you can rent a dedicated usability lab. Here’s what to consider.
Your Own Office Space
A spare conference room or an unused office can be a quiet space for a usability test and can be easily scheduled (although even this can be a hard ask in the stuffed offices in some cities). You can also rent a conference room at a hotel or a temporary offices space, such as WeWork, or even look for a space at a local university.
Noise: If you’re selecting your own space, be sure to consider ambient sound from adjacent offices or street noise. (There’s always a siren or car alarm blaring at the worst time.)
Internet: Wi-Fi seems to stop working at the worst times. Be sure you have reliable Internet in whatever space you choose. Even if you’re testing low-fidelity prototypes, you and your participants will need access to the Internet.
Influence: If you invite participants to a well-branded company campus (think big company logos), they might not be objective.
Lighting: It seems simple, yet sun glare or horrible fluorescent lighting can make it hard to see mobile or desktop screens and can even give participants a headache. Look for something well lit.
Renting a Lab or Facility
Most large cities have dedicated usability labs or focus group rooms you can rent by the day. These dedicated spaces have the lighting, noise, and Internet usually figured out for you (for a price). These typically cost between $1,000 and $3,000 a day and may include software and a technician. This may break your budget (three days can cost $10,000, not including participant incentives).
With a space arranged, consider the software you need to present your interface and record what you observe.
Laptop or device: We recommend having at least separate participant’s and moderator’s laptops to allow you to take notes and mirror a participant’s screen. With only a single laptop it will be challenging to see what participants do and take notes electronically but it’s still feasible.
For mobile studies, most participants are fine using their own devices if they don’t have to download anything. Provide a backup iPhone or Android phone if you have an app prototype.
Screen recording and video editing software: A lot happens during a moderated usability test. If you’re both the facilitator and note-taker, you’ll probably miss things participants say and do. Recording a participant’s screen (with their permission) allows you to review what you saw (or thought you saw).
There are free (open-source) options such as QuickTime and OBS, to paid software such as Camtasia and GoToMeeting that cost around $300 a year. Be sure you try any software before your first session!
Logging software: You can use simple pen and paper or take notes on your computer. There is also software to help you systematize your observations, such as Excel and Google Sheets to more expensive options from Morae and Ovo Studios (both $2500+).
Mirroring software/sled: If participants are using their mobile phones, you may want to mirror their phones using software or use a handheld sled. Mirroring software comes with iOS devices (AirPlay) and there are options to mirror for Android.
Camera audio/video: Recording what the participant is doing physically is especially insightful for mobile or physical device studies. A simple $30 web cam is usually sufficient.
Audio: Built-in mics on most laptops often can’t pick up participants and facilitators adequately. An external web cam usually has a mic that can sufficiently separate voices from the clickety-clack of the keyboard. You can also purchase a high-quality desktop USB mic.
Usability testing is more effective if the stakeholders can observe at least some of the sessions, ideally live. For better or worse, people are more convinced by a concrete story. Watching the ups and downs of one participant struggling to use an interface will often convince more product owners on what needs to be fixed than seeing a graph of metrics for a thousand participants.
As soon as you have observers, you’ll need a way to have them observe without interfering or a way to watch and communicate remotely. If they’re in another location, you can have them join via GoToMeeting or WebEx. These services have built-in chat that usually works well. But if using these programs, be sure observers can’t accidentally chat with participants! We use a separate real-time chat program that only displays on the moderator’s computer to prevent the wrong messages and allows for faster communication.
Conducting a usability test doesn’t have to be expensive or complicated. If you have a limited budget, save your funds for finding and compensating qualified participants. Focus your time on deriving realistic task scenarios that address business questions and uncover problems that can be fixed. If you need a professional setup, renting a lab will cost more money but solve many logistical issues for you. If you conduct regular usability testing, consider building a dedicated space or lab and don’t be afraid to call in the professionals when you need expert support with a short timeline.
Learn More: UX Measurement Boot Camp
Intensive Training on UX Methods, Metrics and Measurement
|Fall 2020: Delivered Online|