UX research efforts should be driven by business questions and a good hypothesis.
Whether the research is a usability evaluation (unmoderated or moderated), survey, or an observational method like a contextual inquiry, decisions need to be made about question wording, response options, and tasks.
But in the process of working through study details, often the original intent of the study can get lost.
At its worst, study-design can get bogged down by internal politics as multiple stakeholders provide input.
Decisions are made to satisfy multiple stakeholders rather than what most efficiently addresses research goals. To help ensure a study design addresses the research questions and to help guide decision-making, we’ve found a grid that aligns research questions to study components helps.
To create a research grid, follow these steps.
- List the research goals and hypotheses. Examples of research questions include:
- What are the pain points users have with purchasing and installation of our enterprise software product?
- Are participants noticing product placements when searching for computers on our website?
- How is our brand perceived by our current and prospective customers?
- How loyal is our customer base?
- Do financial advisors use the product filter to find our mutual funds?
- How accurate are the search results? Do key products appear on the first page of the search results page?
- Place the research goals, questions, or hypotheses at the top of a grid (see the examples below). You can use Excel, PowerPoint, Word, or even a whiteboard.
- Identify how the research goals or questions will be addressed in the research study. These are a combination of:
- Specific questions that are asked, such as in a survey (question numbers can work).
- Observations and artifacts, such as from directly observing a participant or viewing videos, click maps, or heat maps.
- Tasks and post-task questions (if a task-based study).
- Derived insights, such as from a key driver analysis, factor analysis, or through other combinations of questions or analysis (such as creating variables from verbatim comments).
- Look for gaps where you’ll have light or no coverage of the research questions. If a hypothesis or research question has few or no items addressing it, you’ll need to modify your study (such as adding questions) or save the research questions for another study. One study can’t answer all research questions!
- Cut the bloat. Questions or tasks that don’t match up with research questions are candidates for removal (such as excessive demographic questions) or at least may require some justifying.
Two examples of research grids are shown below. Figure 1 was pulled from a customer survey study plan from PowerPoint. Figure 2 shows a matrix from Excel from an unmoderated website study. Both have research questions at the top and then reference how each will be addressed using question numbers, tasks, or derived insights.
The study from the matrix in Figure 1 was primarily focused on understanding current and prospective customer perceptions of a brand and product. Consequently, we were able to use mostly closed-ended questions with rating scales to address the questions (as is common with branding and satisfaction surveys).
The unmoderated study from Figure 2 had questions that were addressed by watching how participants attempted three tasks on a website. This matrix shows how we often combine similar research questions; the fifth set of questions in Figure 2 includes related research questions about banner placement. We used videos and click maps primarily to address these.
It’s all too easy for research studies (especially big ones) to get derailed by details. Using a research grid is one way to help focus decisions and keep both the study design and ultimately the analysis focused on the original intent of the study.
|UX Measurement Boot Camp : Three Days of Intensive Training on UX Methods, Metrics and Measurement Aug. 8th-10th 2018|