Some methods have been around for decades (like usability testing), others are more recent additions, while some seem to be just slight variations on other existing methods.
We’ve been tracking and analyzing the methods UX professionals report using for a few years by analyzing the results of the UXPA salary survey.
The recently completed 2018 UXPA salary survey provides one of the more comprehensive and unbiased pictures of UX method usage and how it’s changed over time. The survey this year contains data from over 1,300 respondents from 51 countries and is comparable to the historical data from 2016, 2014, and 2011, with similarly sized samples.
Here is an overview of how common methods are used and how their use has changed over time. Differences of greater than about 5% across years or methods are statistically significant.
As has been the case since 2011, usability testing is the most-used method, with three-quarters of respondents using it. Usability testing has evolved in the last decade with many more options to provide unmoderated and remote moderated testing, in addition to traditional lab-based testing. It’s not surprising this core method remains popular in 2018.
The term Heuristic Evaluation is often used interchangeably with expert review (much to the chagrin of Rolf Molich). Both expert reviews and heuristic evaluations are analytic techniques that require an evaluator to review an interface against a set of guidelines or principles. This method is frequently used because it’s faster and less expensive than many other methods. 62% of respondents report using expert reviews, which is slightly down from 64% in 2016 and showing a downward trend since 2011.
Benchmarking a website using standardized metrics and against competitors is an excellent tool to understand how design changes impact the user experience. I strongly recommend the approach for tracking interface changes over time. If you need help, I recently wrote a detailed and practical book for running a benchmark study.
Almost half of respondents report benchmarking or running competitive studies (which is good), but usage is down a bit since 2011.
Eye tracking continues to remain a niche method with only 10% of respondents reporting using it. While eye tracking has gotten a lot cheaper in the last decade, its usage is down since we started tracking it in 2011 when 16% reported using it. Its more limited use may reflect the considerable time and training needed to include eye tracking. For example, when we conduct eye tracking studies for our clients, we allot around 10 minutes of processing and analysis time for every one minute of video.
User research is a broad term that encompasses many methods both qualitative and quantitative. Not surprisingly, then, most respondents report doing some form of user research. The percentage has remained relatively consistent since 2011 (see the figure below), with the exception of focus groups, which dropped statistically from 2016 (37%) to 2018 (32%). With focus groups losing traction among UX professionals, it’s not surprising this method continues to lose popularity (which is probably a good thing as other methods are usually more appropriate in UX). Surveys remain a popular UX research method despite some criticism and I suspect they will remain so as they continue to be integrated into other methods like unmoderated studies.
Defining Users & Requirements
Understanding who the users are and what they’re trying to accomplish with an interface can be essential to improving the user experience. Personas have remained a popular method for defining users despite criticism of how they’re implemented. It’s not surprising then that one of our most popular articles is on how to make them more scientific.
The number of respondents gathering requirements and conducting a contextual inquiry also dropped slightly from 2016. Task analysis continues to show a decline with its usage dipping from 52% in 2011 to 39% this year.
We’ve consistently found that the ability to find products and information easily is what differentiates good websites and products from bad ones. There’s a science to findability and it involves a number of methods and techniques. The UXPA survey asks about one popular method, card sorting (which is included in our MUIQ platform), which has remained relatively popular since we began tracking it in 2011.
Prototyping & Design
Design is a fundamental part of the user experience. Not surprisingly, more than 70% of respondents report using some form of prototype or wireframe. We test high-fidelity prototypes built from InVision quite frequently on desktop and mobile in our MUIQ app. We’ve also found UX metrics to be reliable, even when used on these early-stage prototypes.
Accessibility remains a more niche activity for this group of respondents but there seems to be a steady growth in assessing accessibility since 2011. In the past seven years, it’s increased from 14% to 21% (which is also a good thing).
Strategy, Training, & Design Thinking
Strategy, project management, and training in UX have been consistently common activities since 2014, with more than a third of participants doing some project management, training, content strategy, and strategic consulting each year. The most common activity remains conceptual design, which is even showing a slight increase in 2018 compared to 2016.
The results of the 2018 UXPA salary survey reveal that the core methods in UX remain popular today, including usability testing, expert reviews, personas, card-sorting, and prototyping. Some methods like focus groups and eye tracking have continued to lose popularity over the last seven years, while accessibility reviews have increased in usage.
Learn More: UX Measurement Boot Camp
Intensive Training on UX Methods, Metrics and Measurement
|Denver: Aug. 5th-7th, 2020|