Topics
Topics
Is UX Data Normally Distributed?
If you took an intro to stats class (or if you know just enough to be dangerous), you probably recall two things: something about Mark Twain’s “lies, damned lies …,” and that your data needs to be normally distributed. Turns out both are only partly true. Mark Twain did write the famous quote, but he
A Guide to Task-Based UX Metrics
When quantifying the user experience of a product, app, or experience, we recommend using a mix of study-level and task-based UX metrics. While it’s not always feasible to assess a task experience (because of challenges with budgets, timelines, or access to products and users), observing participants attempt tasks can help uncover usability problems, informing designers
Difficult–Easy or Easy–Difficult—
Does It Matter?
The seven-point Single Ease Question (SEQ®) has become a standard in assessing post-task perceptions of ease. We developed the SEQ over a decade ago after our research showed it performed comparably to or better than other single-item measures. It is an extension of an earlier five-point version that Tedesco and Tullis (2006 [PDF]) found performed best
Sample Sizes for Comparing SUS Scores
Microsoft Word is a widely used word processing program, part of the Microsoft Office suite of programs. While its dominance has been challenged recently by Google Docs, Word still leads on the features list, providing many features that Google’s offering lacks. But adding features can also add to bloat, making common tasks harder as users
A New Statistical Approach for Predicting Usability Problems
In an earlier article, we described the most common methods for modeling the total number of unique usability problems uncovered in a usability test: the average problem occurrence (p), adjusted problem occurrence (adj-p), beta-binomial, and specific problem probabilities. While these methods provide reasonably accurate predictions of the total number of unique problems, there is still
UX and Net Promoter Benchmarks of Grocery Websites
Online grocery shopping has increased substantially over the past few years due to the COVID-19 pandemic. In 2019, online grocery sales grew steadily, but they accounted for only a modest portion of sales. Now, online grocery sales are booming, with estimated annual sales of 97.7 billion in 2021, an impressive 57% increase for the market
How to Report Product Experience Data
How often do you use your phone? How about a streaming service you subscribe to, like Hulu or Netflix? What about software, like Microsoft Excel? In an earlier article, we discussed the importance of measuring prior experience. Prior experience is a strong predictor of UX metrics. People who are familiar with a product and use
What Is the Product-Market Fit (PMF) Item?
A social media platform, a GPS heads-up display, a video streaming platform: all are examples of viable products … that failed. The failure came despite often substantial financial funding and putatively useful features. Their failures were blamed on poor product-market fit. Predicting a prospective product’s success in the market isn’t easy. But there’s a strong
Eight Laws of Statistics
Statistics doesn’t have a Magna Carta, constitution, or bill of rights to enumerate laws, guiding principles, or limits of power. There have been attempts to articulate credos for statistical practice. Two of the most enduring ones are based on the work by Robert P. Abelson, a former statistical professor at Yale. If Abelson wasn’t the
Sample Sizes for Comparing SUS to a Benchmark
The System Usability Scale (SUS) has been used in industrial user experience research since the mid-1980s. Despite its age, the SUS is still a popular measure, widely used in benchmark tests of software products to measure perceived usability. One reason for its popularity is the extent to which its measurement properties have been comprehensively studied
Why Collect Task- and Study-Level Metrics?
In Quantifying the User Experience, we recommend using a mix of task-level and study-level metrics, especially in benchmarking studies. But what, exactly, are task-level and study-level metrics, how do they differ, and why should you collect them both? In this article, we’ll explore this common practice of collecting both types of metrics to understand the
Five Styles of Statistical Rhetoric
When learning statistics, you’ll encounter many formulas based on principles of probability and mathematics. But statistics isn’t just a formulaic process where you enter data and are told what to do. Statistics should guide, not dictate, decisions. In making decisions, though, there are different styles of interpreting data. Although a lot of people think statistics