The F-Word in User Experience

Jeff Sauro, PhD

“We want to conduct a focus group to improve our software.”

It’s something I hear a couple times a month, and almost always when I’m speaking with a product manager, marketer or someone unfamiliar with usability.

It’s so common that we UX professionals have wryly come to call it the “F-word,” a phrase I first heard from Christian Rohrer, now VP of Design, Research, and Enterprise Services at Capital One.

We call it the F-word not because focus groups are inherently bad but because, in the context of understanding the user experience, they’re usually the wrong tool for the job.

Traditionally, focus groups have been used in marketing and advertising as a means for obtaining feedback about new products or services. Like a scene out of Mad Men, focus groups have been part of the discourse for decades. So it makes sense when a marketing manager uses the lexicon she is familiar with.

Focus groups provide rich and detailed information about feelings, thoughts and underlying emotional motivation from people in their own words. In short, focus groups focus on attitude and affect.

Advantages of Focus Groups

Like many in the field I have a knee-jerk negative reaction when I hear the F-word. But I do see two major advantages of focus groups:

Efficiency: You can collect input from many people in a shorter amount of time. In as little as 60-90 minutes you can collect input from 8-12 people.

Group Dynamics: The unpredictability and fluidness of conversations between strangers can generate novel ideas and illuminate core emotional motivations behind choices and preferences. Some people are better articulating these ideas and getting cues from others stimulates their ability to express their desires and motivations.

For example, brainstorming new product ideas, website features, new flavors or tastes are good for the group to toss around and build upon.

Or why will some people spend a lot more for a premium brand? The group setting is a therapeutic event that helps uncover the core emotions that lead to product preferences.

If you understand the root cause of fickle preferences you can speak to that emotion in marketing, advertising, and product aesthetics.

I didn’t include “cheap” as one of the advantages of focus groups even though they are often perceived as the most cost effective way to collect consumer insights.

Even a quick focus group with 6-8 participants and a trained facilitator will cost thousands of dollars when you factor in the honorariums, facilitator time and the real cost—converting the conversations into interpretable and actionable data.

Disadvantages of Focus Groups

The focus group’s biggest advantage “the group” is also its biggest shortcoming. It leads to a few methodological disadvantages.

Dominance: Talking to many people at once means invariably some will dominate the conversation; others will tacitly agree or simply be unnaturally influenced by others.

Group-Think: People in groups may coalesce unnaturally in agreement on an idea or preference. Would they have agreed if they were around another group, with their family or by themselves?

Social Desirability: Most people don’t want to offend or hurt other people and can be more reserved in articulating their true feelings, favoring instead to express more acceptable opinions.

Non-Independence: Just because you had 12 people in a room doesn’t mean you can treat data collected like it were a sample size of 12. The influence each person has on even binary choices (do you prefer A or B) can make even simple computations suspect.

These four disadvantages together mean I treat insights from focus groups as tentative. It’s probably more accurate to call it the “G” word.

Are the preferences, emotions and desires really held by users or are they an artifact of the group? When possible I like to subject ideas or conclusions from focus groups to another level of scrutiny with a survey or individual session with users.

Usually we don’t have time to do both a focus group and follow up session, which leads to our recommended approach.

The 1:1 Session / Usability Test

I am a champion of usability testing but know it’s not a panacea. I understand the limits of putting a design with closed-ended tasks in front of a user. Many UX professionals also understand that there’s a place for focus groups and often make the distinction between focus groups (what marketing does) and usability testing (what the product team does).

At the recent UxPA conference in Las Vegas, panelists on the “Usability Engineers Do Focus Groups Too” also articulated the difference between focus groups and usability testing. It’s common to think of the two methods as distinct options but I see it as a false dichotomy and think many of the benefits of focus groups can be applied to a usability test.

In fact, one panelists, Jen McGinn, emphasized that the two methods have much in common: “You use a script, you recruit to user profiles, you put the most important questions up front in case you run out of time, and you recruit the same way.”

We also see that similarity but instead of running focus groups we combine what we find are the best of focus groups and usability testing into a 1:1 session.

At the beginning of our session we probe an individual’s motivations, the goals, unmet needs and behaviors as related to the website or product. After a more open-ended discussion we have ready some tasks and interfaces to work though. It’s like a focus group with one person.

For example, we recently conducted an evaluation to improve the usability of a mobile credit card website. Part of the project was to understand what needed to be fixed, but also to understand what functionality is missing and why many cardholders weren’t using it.

Most of our participants reported having never paid any credit card bill on their mobile phones. At the beginning of each session we spent a few minutes with each participant to understand why they hadn’t. Some participants would immediately say it was “security concerns” while others would provide examples of news stories of credit card information being “skimmed through the air.”

We didn’t have a closed set of multiple choice options but in many different ways we heard users expressing the same core emotion of fear.

Their primary reason for not using the mobile website wasn’t the usability, it was security. These sentiments were expressed by individuals and not seeded by others in a group. It allowed us to quantify the qualitative comments and make some generalizations about the entire user population with just a few users. Knowing this core reason, along with its likely prevalence, convinced the product team that messaging might have a bigger impact on usage than usability.

Just because you are conducting a usability test doesn’t mean the entire session needs to be some sterile scripted sequence of perfunctory motions. Make the most of the recruiting effort and time with the user. The ability to combine emotions and motivations with actual task behavior in a 1:1 session provides a better view into the user experience.

There is a place for focus groups in UX research. But the next time you hear the F-word consider keeping the focus and losing the group.

Here are some more resources on usability testing and focus groups:

 

0
    0
    Your Cart
    Your cart is emptyReturn to Shop
    Scroll to Top