Topics
Topics
Assessing Interrater Reliability in UX Research
Do researchers agree on what the problems are in an interface? And will researchers group the problems into the same categories? When coding open-ended comments in a survey, will different researchers classify the comments differently? These discovery and classification activities are quite common in UX research, but they are often conducted by a single person
10 Key Takeaways from the Latest Research on Thinking Aloud in Usability Testing
In Think Aloud (TA) testing, participants speak their thoughts while attempting tasks. The process is meant to help researchers identify usability problems and potential fixes. It’s a distinctive method in UX research. Despite its popularity, there are many open research questions about the efficacy and potential side effects of think-aloud research. Researchers still disagree on
49 UX Metrics, Methods, & Measurement Articles from 2023
All of us at MeasuringU® wish you a Happy New Year! In 2023, we posted 49 articles and continued to add features to our MUiQ® UX testing platform to make it even easier to develop studies and analyze results. We hosted our eleventh UX Measurement Bootcamp, again as a blended virtual event with a combination
Comparing SEQ and Click SMEQ Sensitivity
Capturing someone’s attitude as precisely as possible with as little effort as possible … that’s the goal of post-task metrics collected in usability tests. Organizations devote time and money to testing products with users, not to watching users spend time reading and answering questions. Organizations want to understand if people think an experience is difficult,
An Experiment Comparing 2D and 3D Bar Graphs
You can make graphs in 3D. They change up often banal-looking flat graphs. But is it more than just an aesthetic decision? Does adding a third dimension to bar graphs make it easier or harder for people to interpret? In an earlier article, we reviewed the literature on the use of 3D graphs. There are
Frameworks for Classifying UI Problems
Finding and fixing problems is a core activity of much of UX research (similar to identifying and preventing software bugs and product defects). The problems found while users attempt tasks are often broadly referred to as UI problems because the friction points tend to involve issues that blur the lines between bugs, functional deficits, and
How Do Changes in Standard Deviation Affect Sample Size Estimation?
The standard deviation is the most common way of measuring variability or “dispersion” in data. The more the data is dispersed, the more measures such as the mean will fluctuate from sample to sample. That means higher variability (higher standard deviations) requires larger sample sizes. But exactly how much do standard deviations—whether large or small—impact
Sample Sizes for Comparing Rating Scale Means
Are customers more satisfied this quarter than last quarter? Do users trust the brand less this year than last year? Did the product changes result in more customers renewing their subscriptions? When UX researchers want to measure attitudes and intentions, they often ask respondents to complete multipoint rating scale items, which are then compared with
Should You Report Numbers or Percentages in Small-Sample Studies?
“Don’t include numbers when reporting the results of small-sample research studies!” “If you must, definitely don’t use percentages!” “And of course, don’t even think about using statistics!” We regularly hear variations of this advice from well-intentioned researchers, often senior ones. In 2005, we encountered this debate among UX professionals when we participated in a workshop
UX and NPS Benchmarks of Hotel Websites (2023)
Planning your next vacation can be both exciting and overwhelming. Most travelers do their research and make reservations for flights, activities, and accommodations online. In particular, hotel websites offer the convenience of browsing, comparing, and booking hotels from anywhere. Hotel bookings and revenue have recovered from the depths of the pandemic. Yet with all this
Sample Sizes for Comparing Dependent Proportions
Sample size estimation is an important part of study planning. If the sample size is too small, the study will be underpowered, meaning it will be incapable of detecting sufficiently small differences as statistically significant. If the sample size is too large, the study will be inefficient and cost more than necessary. A critical component
Are People Who Agree to Think Aloud Different?
In an earlier article, we showed that only about 9% of panel participants will eventually complete a study in which they are asked to think aloud. That is, if you need ten usable think-aloud videos, expect to invite around 111 participants. On the surface, this means you’ll need to plan for a lot of people