Few can track how money invested in improving the user experience translates into profitability.
By quantifying design efforts and outcomes all organizations can benefit from understanding how improving the user experience can improve revenue.
The Engine of Growth
If you can imagine revenue being the final gear in a business machine designed to produce profits, we can think of the smaller gears as all the possible variables that crank out the revenue. While there are a large number of possible variables: competition, branding, the economy, price and perceived worth to name a few, there are usually a few critical variables that disproportionally explain a product’s success or failure.
While there’s not a one-size fits-all formula, I have found perceptions of utility, quality, value, usability and loyalty to have the most explanatory power after examining survey and revenue data from several software products and websites.
The green gears are the outcomes that really matter for business investments. I’ve segmented them into short-term and long-term profits because many business decisions can cut-costs or boost sales in the short-term but have a deleterious effect on long term viability (e.g. cutting support or advertising).
The blue gears in the figure are measured from closed-ended survey questions. In more detail they are:
Utility: The gateway driver. If your product doesn’t do something with perceived value the gears don’t move.
Quality & Value: Once there is perceived value you have to deliver a consistent and high-quality product at the right price to continue generating revenue
Usability: Perceived usability turns with quality and value in driving both short-term revenue and as a key driver of customer loyalty.
Loyalty: Typically measured using the Net Promoter Score, such measures of the customer experience are leading indicators of revenue. The NPS isn’t a perfect metric but some research has established it as a reasonable link to long term growth. Many companies now use it as a proxy for revenue and pay bonuses on how much it increases.
Short & Long Term Revenue: For products with short purchase cycles and repeat purchases (e.g. e-commerce websites) it is easier to measure revenue directly. For products with longer purchase cycles or one-time purchases, changes in revenue are better measured as increases or decreases in maintenance contracts or upgrades.
The gray gear represents unmeasured variables or difficult to quantify factors that are either out of our control or that we are unaware of. It reminds us that even a systematic process of quantifying inputs cannot perfectly predict future outcomes.
Separating the critical few variables from the trivial many that best explain a product’s success or failure is determined using Key-Driver Analysis.
Key-Driver Analysis uses Multiple Regression Analysis to mathematically determine which variables predict your outcome variable the best (using standardized beta-coefficients) and they provide the relative importance of each variable.
Typically the outcome variable is the likelihood to recommend question (as used in the NPS) as a proxy for revenue and the predictor variables are questions that ask about usability, value, quality, utility and a potpourri of other product and service questions.
For a variable to be a key driver means it needs to add information not contained in the other variables. I have consistently found this to be the case with ease of use questions. Customer attitudes about how difficult or easy a product is tend to go beyond their attitudes about quality, value, functionality and price.
The gears above are shown in approximate proportion to their contribution. This means that utility, quality and value are typically bigger contributors that usability. Despite being a lessor contributor, there are typically a few easy-to-identify fixes that can move this gear.
This is one of the benefits of a Key Driver Analysis, instead of prioritizing based on company politics or intuition, you use data from your customers.
Moving the Gears
Each gear in the machine represents an outcome that can’t be moved directly. Instead, you have to change the product, service, customer support, pricing, features or other factors to move the gears.
Correlation is of course not causation. So while it is simpler to think in terms of the gears moving in one-direction to drive profits, it is also likely that higher revenue from successful products cast a Halo on attitudinal metrics. That means high satisfaction scores could just be a result of a successful product and not a cause of the success.
So while there are flaws with this model (as there are with all models) I find it useful for prioritizing tactical activities to support long term strategic goals and helping establish a program for measuring ease of use and ultimately tracking the return on UX investments.
Moving the Usability Gears
Reductions in the number of interaction problems, time to complete tasks and increases in task completion rates are examples of metrics tied to tactical activities that ultimately move the usability gear. These changes can be tracked using a questionnaire like the System Usability Scale (SUS) or Standardized Usability Percentile Rank (SUPR-Q) which are loosely tied to task-performance metrics and user-interface problems.
You can think of these tactical changes as very-small gears inside the usability gear. By making small incremental changes over-time and measuring the movement in SUS scores, these small movements to the gears will help drive the engine of profits.
Measuring the Impact of Usability Investments
Here’s the minimum you need to do to get started measuring the impact of usability on your product:
- Measure Revenue or another key business metric: repeat business, renewed maintenance agreements or new customers. If you don’t have access to revenue data find a good proxy like Net Promoter Scores (ideally use both).
- Survey customers about their attitudes around product features, customer support and services. Be sure to include questions about usability, value and quality–keep it as short and simple as possible.
- Conduct a Key-Driver Analysis to identify which attributes have the biggest impact on loyalty and revenue. Usability will likely be one of your key-drivers and something you can take actionable steps to improve. Despite asking your customer many questions, you’ll be surprised how just a few attributes predict most of the changes in loyalty.
- Track UX inputs and outputs: There is always a long list of fixes and enhancements. Instead of picking the oldest or easiest ones to implement, identify the high-impact changes by examining the verbatim comments from the survey responses. Then setup metrics to track the positive impact on design changes. Finally, track the time and money spent on testing users, conducting site-visits and other UX activities.
Following these steps allows you to more effectively prioritize your UX efforts and establish a link to revenue. Contact Measuring Usability LLC for help getting started surveying your customers, conducting a Key Driver Analysis or establishing an effective system for measuring your product’s user experience.
Learn More: UX Measurement Boot Camp
Intensive Training on UX Methods, Metrics and Measurement
|Fall 2020: Delivered Online|