But if you or your organization conducts more than the occasional usability test, which it probably should (another topic in itself), you may want to consider setting up a dedicated usability lab.
Having a dedicated space for testing is a hallmark of organizations with high UX maturity. Organizations rated as mature in UX are almost twice as likely to have a dedicated usability space as those rated as nonmature (see Figure 1). Mature UX organizations are also three times more likely to use a one-way mirror in usability labs.
“Lab” is of course a fancy way of saying an office or room—not the image of chemicals and electroshock treatment that the word “lab” probably conjures up.
We conduct usability tests every week and can have hundreds of participants in our labs each month. We needed dedicated space and the right technology to support this amount of usability evaluation.
Having recently built and installed two new labs in our offices in Denver, CO, I can tell you a bit about the process to save you some of the headache. You should plan a sizable budget for technology and a space you can customize with lots of wires. It takes both planning and a little improvising.
You’ll need a space that’s big enough to accommodate two to three people, and it should be next to another office that can be used as an observation room (with an optional one-way mirror).
The observation room often needs to be as big as or bigger than the lab and will need to accommodate five to ten observers. The space you have available may dictate the size, but for reference, our labs are about 12 by 14 feet and our observation rooms are between 12 by 10 feet and 16 by 10 feet (see Figure 2).
In the usability lab you should have comfortable chairs for the moderator (they have to sit there all day!), participant, and note taker. If you test mobile or physical products such as remote controls or TVs, consider having a comfy couch as well (Figure 3a).
A lot of usability labs I’ve seen are buried in basements or interior spaces without windows. This is usually good for soundproofing and easy access for participants. If you spend a lot of time in the lab though it can feel like a prison cell (even solitary!). If you have a lab with windows as we do, you’ll need blackout curtains (or double curtains) to manage any light glare or bleed. Fortunately, we’ve found that a little natural light makes everyone cheery, and it doesn’t interfere with the study. (And participants don’t mind a view of the mountains!)
Blackout curtains are especially important in your observation room if it has windows (Figure 4), as the light from the observation room can be seen through a one-way mirror.
You’ll need plenty of outlets in your lab and observation room. If you want ceiling cameras, many require external power, so plan to have power installed in the ceiling.
Many offices have drywall that goes only as high as the acoustic tiles. Above the lights and tiles is often open space between rooms, meaning voices will carry easily. You need to get insulation installed all the way to the ceiling (often called the hard-deck). It’s not always easy or cheap but insulation makes a huge difference for reducing sounds (especially when you have a one-way mirror).
The visual symbol of the traditional usability lab is the glass the observer can see through and the participants generally can’t.
One-way mirrors are custom pieces of glass, often with a reflective film, that require custom installation. You’ll want high-quality double-paned glass that offers soundproofing. Our labs are so soundproof that you have to shout to hear through the mirrors. To communicate more effectively than shouting, install audio equipment instead.
Place for Participants
Like a doctor’s office, participants can rarely just show up and be “seen” in the usability lab. Having some sort of waiting room away from the lab will help. This is particularly helpful when you’re running many participants through in a day and dealing with some participants that show up early, others late, or sessions that run over. We have a reception area with magazines, water, and snacks for participants to wait in (usually not for very long). We’ve had as many as 50 participants through our labs in a day, so this waiting area is essential.
Plan for the following for your space and furniture:
Low: $1,000 for cheap and less furniture, no observation room, and no mirrors
High: $75,000 for nice chairs, couch, curtains, observation room, and a one-way mirror
A good usability lab needs to pick up both what the participant says and what the facilitator says. This usually requires multiple microphones and equipment to support them.
We experimented with different directional and desktop microphones and found a combination of a sensitive but discrete ceiling microphone and a tabletop boundary mic work best. This allows the moderator (or controller) to adjust the audio level for a quiet participant without increasing the volume for the facilitator.
Push to Talk/Voice of God
The people in the observation room will need the ability to communicate with the people in the lab—especially if the facilitator is set up in the observation room (a practice that seems to have lost favor). The best microphone is one that requires an explicit action (push to talk) and isn’t easy to leave on. You don’t want to have to worry about having a “hot-mic” incident where participants hear the conversations of the observers.
Speakers and Mixers
To listen to the audio from an observation room you’ll need to have speakers (usually powered) with an audio mixer and amplifier. These can get quite expensive and require separate audio racks and programming from an audio technician. We’ve found that two separate powered speakers connected with separate amplifiers offer a low cost, but effective, solution. While we’re all about the latest technology, having a simple physical dial and control to adjust the volume (or quickly lower it) is easiest for observers who aren’t familiar with your setup. I’ve heard of people using baby monitors as a very cheap option for monitoring. We haven’t tried that option but found cheap intercoms to be insufficient to communicate between rooms.
Watch out for feedback. We were warned about having microphones and speakers too close, generating possible feedback. High-tech equipment has built-in echo/feedback cancellation but can quickly increase your costs as you’ll need a sound engineer to configure. The last thing you want are those annoying feedback squeals in the middle of a session. Test your audio setup before committing.
Low: $2,000 for powered speakers, amp/mixers, and good microphones
High: $20,000 for a programmable bi-amp with echo cancellation, programming, and digital audio/Crestron controls
The difference between a high-quality digital camera and a consumer web cam is noticeable. Look for something with high resolution that allows you to control focus and zoom.
Ceiling cameras are ideal for mobile or hardware studies where you want to see a birds-eye view of the participants. Ensure the cameras have at least 10x zoom and focus control to see details even on small handsets.
Technology sometimes just demands more technology. If you have cameras in the ceiling, you’ll need a way to control them. You’ll want a PTZ (pan tilt zoom) camera to allow you to focus and zoom (usually on mobile devices) and most PTZ cameras come with their own remote controls. The fancier ones use something called VISCA control, which allows you to control them with neat joystick-like controls (Figure 8 above). Most PTZ cameras require external power, so you’ll need to be sure you have power run above your ceiling tiles (or you’ll need to look into more expensive power over Internet cameras).
Low: $100 for a high-res web camera
High: $10,000 for cameras, control cabling, and installation
Computer and Software
All those camera feeds will need to go into a computer with enough power. We use a tower with a lot of RAM and storage and a great graphics card. Something you’d use for gaming.
Screen Recording and Mirroring: Camtasia and Morae are popular commercial programs that will record a participant’s screen. We use a solution called Open Broadcaster Software, which is open source and offers much more flexibility.
Streaming and Communicating: If you have observers in another room (or location) who need to watch the sessions live, you’ll need a license for GoToMeeting, Zoom, WebEx, or other web meeting software. These programs also allow you to chat with the facilitator—be sure you have it configured so observers’ messages don’t pop up on the participant’s screen! While the chat in these streaming services work fine, we use a separate real-time chat program that allows the facilitators to see letter-by-letter messages from observers. Things happen so quickly during a usability test that by the time an observer types a message, the moderator might have already moved on.
Don’t rely on flaky Wi-Fi. Be sure you have a direct high-speed connection to the lab and save the Wi-Fi for participants’ mobile devices or other non-essential items.
Low: $1,000 for a good laptop and free mirroring/screenshare software
High: $10,000 for a high-powered tower with commercial software and screenshare/chat
Total Costs and Time
In addition to the costs for the main parts described above, you should anticipate the additional cost and time for wiring, installation, and drywall modifications. Through the process be sure the technology is documented so researchers can easily use it and troubleshoot problems that will inevitably occur (for example, loss of sound, audio, or screen sharing).
Low: $5,000 for the basics of cameras, computers, and audio
High: $150,000 for a custom build-out with one-way mirrors and high-end options
The cost and time associated with building and maintaining a dedicated usability lab may not make sense for all organizations. This is why many outsource some or all of their usability testing to research firms like MeasuringU. External vendors can extend the capacity of overworked research teams by helping with recruiting, providing dedicated labs, and assisting with the facilitation. But just because you outsource the testing, don’t outsource all the thinking that needs to go into developing the right tasks and the importance of getting stakeholder buy-in and ensuring the results are put to good use!
Thanks to our friend Dean Barker and his team at Optum for providing some insight into their lab-building experiences.
|UX Measurement Boot Camp : Three Days of Intensive Training on UX Methods, Metrics and Measurement Aug. 8th-10th 2018|