Management often gets blamed for not appreciating or understanding the importance of UX.
While managers can’t be expected to fully grasp all details of UX research and design, if they (or you) are going to effectively manage the efforts, you should know best how to measure them.
Here are ten key concepts for managing and measuring UX.
1. Adopt a measurement plan.
A good UX measurement plan starts with identifying company Key Performance Indicators (KPIs) and linking UX metrics and activities to those KPIs. Understanding which UX activities move the corporate needle helps prioritize the product areas, methods, and metrics that provide the best bang for the buck.
2. Identify top tasks.
Users “hire” software, websites, and products to do a job—finding products online, filing taxes, or onboarding a new hire. While most interfaces can support hundreds to thousands of features and tasks, users use, purchase, and recommend based on a few critical tasks. Be sure you and the UX team know what these top tasks are and measure how well users can accomplish them.
3. There is no single UX measure.
There isn’t a magic formula or one metric that best quantifies the user experience. Instead, you need to combine what users do with an interface (actions) and what users say about their experiences (attitudes). Measurement can be obtained from actual use or simulated use of an interface (usually a product or website).
4. What users say and do isn’t always the same.
Users’ attitudes affect their actions, but not always in predictable ways. What people say, or how they respond to questions about an interface, isn’t always the same as how they use it. For example, users might prefer one design but perform better on another (such as complete tasks faster or more effectively). You need to measure both attitudes and actions and reconcile the differences. We typically defer to what people do over what they say.
5. Use multiple methods.
There are multiple methods to measure the user experience. The most common methods involve collecting data from participants in a one-on-one setting. Data can be collected from simulated use (in a usability test), from recalling past use (in a survey or in-depth interview), and from observing actual use (direct observation, log files, and A/B testing). Using a combination of these methods is the best approach.
6. Focus groups are rarely the method to use.
While they have played a pivotal role in the history of market research and are a regular staple of TV shows, focus groups tend not to be the best method for evaluating experiences. To prevent group think and design by committee, it’s better to involve users in one-on-one sessions using usability tests, observing actual usage, or from interviewing.
7. You can assess UX without users (if you have to).
Ideally interfaces are measured and evaluated using samples of actual users. Sampling isn’t always possible due to cost or difficulty obtaining participants. The next best option is to generate proxy measures of interface experiences from Heuristic Evaluations/Expert Reviews, Keystroke Level Modeling, and PURE evaluations, all which can provide measures of an experience.
8. Measures of UX correlate with measures of loyalty.
UX measures, such as the System Usability Scale (SUS), explain around 30-50% of the variation in the Net Promoter Score. Consequently, it’s likely that improving the user experience will improve customer loyalty and ultimately revenue.
9. Compare to benchmarks.
Part of a good measurement plan is instituting a repeatable benchmarking process. One of the best ways to make any measure meaningful is to compare it to external benchmark reports. For example, compare your scores to the average SUS score or use the SUPR-Q for websites.
10. Assess and improve the maturity of your UX org.
It can be difficult to know how to best organize researchers, designers, and developers. Some organizations evaluate maturity better than others. It can be helpful to assess how well your organization’s processes, methods, and team members support the innovation needed for better user experiences.