All Articles

Paraphrasing the statistician George Box, all models are wrong, some are useful, and some can be improved. In a recent article, we reviewed the most common way of modeling problem discovery, which is based on a straightforward application of the cumulative binomial probability formula: P(x≥1) = 1 - (1-p)n. Well, it’s straightforward if you like playing around with these sorts of formulas like Jim and

Read More

Are people more likely to select response options that are on the left side of a rating scale? About ten years ago, we provided a brief literature review of the published evidence, which suggested that this so-called left-side bias not only existed but also was detected almost 100 years ago in some of the earliest rating scales. Across the publications we reviewed, the effect size

Read More

Finding and fixing problems in an interface is one of the fundamental priorities of a formative usability test. But how many users should you test with? And how many usability problems are there to be uncovered? These questions have been discussed and debated for decades. Early work on problem discovery suggested that the first few users will uncover most of the common problems. This isn’t—or

Read More

The COVID-19 pandemic has led to significant changes in how people have vacationed in 2020. To get away from it all without spending time in crowded places, vacationers have turned to vacation rental websites and have planned longer stays. For example, Airbnb recently reported a year-to-year doubling of long-term (>28 days) rentals and a shift from urban to rural stays. These changes make sense given

Read More

It seems like every few years a new standardized UX measure comes along. Standardization of UX measurement is a good thing for researchers and practitioners. Having common methods and definitions helps with objectivity, generalization, economy, and professional communication. At MeasuringU, we pay a lot of attention to the continuing evolution of standardized UX measurement. The UMUX-Lite is a relatively recent measure that has been attracting

Read More

Errors can provide a lot of diagnostic information about the root causes of UI problems and the impact such problems have on the user experience. The frequency of errors—even trivial ones—also provides a quantitative description of the performance of a task. The process of observing and coding errors is more time-consuming and dependent on researcher judgment than recording task completions or task times. Consequently, errors

Read More

There are a lot of opinions about the best formats for agreement scales. Sometimes those opinions are strongly held and can lead to lengthy, heated discussions within research teams. When format differences affect measurement properties, those discussions may be time well spent, but when the formats don’t matter (or matter very little), the time is wasted. That’s why we have an ongoing research goal to

Read More

Somewhat agree, very satisfied, extremely likely. The labels used on the points of rating scales can affect responses in often unpredictable ways. What’s more, certain terms can get lost in translation when writing surveys for international usage. Some terms may have subtly different meanings, possibly making cross-cultural comparisons problematic. While numbers are universally understood and don’t need translation, does "5" on a seven-point scale properly

Read More

We typically recommend small sample sizes (5–10) for conducting iterative usability testing meant to find and fix problems (formative evaluations). For benchmark or comparative studies, where the focus is on detecting differences or estimating population parameters (summative evaluations), we recommend using larger sample sizes (20–100+). Usability testing can be used to uncover problems and assess the experience. Many usability tests will play both roles simultaneously, formative

Read More

Decisions should be driven (or at least informed) by data. Raw data is turned into information by ensuring that it is accurate and has been put into a context that promotes good decision-making. The pandemic has brought a plethora of COVID-related data dashboards, which are meant to provide information that helps the public and public officials make better decisions. With the pressure to report data

Read More