feature image

Should You Report Numbers or Percentages in Small-Sample Studies?

“Don’t include numbers when reporting the results of small-sample research studies!” “If you must, definitely don’t use percentages!” “And of course, don’t even think about using statistics!” We regularly hear variations of this advice from well-intentioned researchers, often senior ones. In 2005, we encountered this debate among UX professionals when we participated in a workshop

Read More »
feature image with hotel bed

UX and NPS Benchmarks of Hotel Websites (2023)

Planning your next vacation can be both exciting and overwhelming. Most travelers do their research and make reservations for flights, activities, and accommodations online. In particular, hotel websites offer the convenience of browsing, comparing, and booking hotels from anywhere. Hotel bookings and revenue have recovered from the depths of the pandemic. Yet with all this

Read More »
Feature image with bargraph, user icon, and formula

Sample Sizes for Comparing Dependent Proportions

Sample size estimation is an important part of study planning. If the sample size is too small, the study will be underpowered, meaning it will be incapable of detecting sufficiently small differences as statistically significant. If the sample size is too large, the study will be inefficient and cost more than necessary. A critical component

Read More »
Feature image of two women talking

Are People Who Agree to Think Aloud Different?

In an earlier article, we showed that only about 9% of panel participants will eventually complete a study in which they are asked to think aloud. That is, if you need ten usable think-aloud videos, expect to invite around 111 participants. On the surface, this means you’ll need to plan for a lot of people

Read More »
feature image with ranking

How Hard Is It to Rank Items in Surveys?

Ranking questions are a popular way to understand how respondents in UX research prioritize items such as product features, habits, purchases, color schemes, or designs. Forcing participants to make tradeoffs on what’s most important versus least important helps avoid the “everything is important” problem you get when you ask respondents to simply rate how important

Read More »
blog feature image with curve and chart

How to Compare Two Dependent Proportions

In math class, we spend a lot of time learning fractions because they are so important in everyday life (e.g., budgeting, purchasing at the grocery store). Fractions are also used extensively in UX research (e.g., the fundamental completion rate is a fraction), typically expressed as percentages or proportions. Unfortunately, fractions are also hard to learn,

Read More »
blog feature image with paint roller

UX and NPS Benchmarks of Home Paint Websites (2023)

Watching paint dry might be the definition of boring, but the process of picking a paint color and actually painting usually isn’t. Shopping for paint, however, can be frustrating. One early pain point is picking the right brand and color of paint. The process of selecting a paint often starts online at paint websites. Paint

Read More »
feature image with microphone and pointing at laptop

Does Thinking Aloud Reduce the Evaluator Effect?

In Think Aloud (TA) testing, participants speak their thoughts while attempting tasks. The process is meant to help researchers identify usability problems and potential fixes. Indeed, in an earlier analysis, we found an increase in problem discovery. Our evaluation of 153 videos, split between TA and non-TA, revealed that evaluators uncovered 36–50% more problems with

Read More »
featured image with 3d graphs

Are 3D Graphs Always Worse Than 2D Graphs?

There are many ways to visually display quantitative information. Excel offers dozens of chart types and color combinations, including those in 3D. But is it good practice to use 3D graphs? Edward Tufte is a famous and vocal critic of using 3D elements or any other decoration in graphs. In his book, Visual Display of

Read More »
feature image of computer mouse and arrows

Essential Metrics for Click Testing

Click testing is an efficient UX research method for understanding where people click on an image. In an earlier article, we reviewed when and why to use a click test. It is often used in the design and release phases of product development, and it generates mostly quantitative data. We also showed how click testing

Read More »
feature image with computer chips and checklist

Classifying Tech Savviness Levels with Technical Activity Checklists

In an earlier article, we demonstrated the validity of measuring tech savviness with technical activity checklists (TAC™) by analyzing the correlation between TAC scores and successful completion rates in four usability studies. The TAC scores significantly correlated with success rates (i.e., people with higher levels of tech savviness tended to complete more tasks). On average,

Read More »
Featured image

How Have UX Job Titles Changed in the Last 15 Years?

What we do on the job can change. Jobs change and job titles change. Sometimes the titles change more than the jobs. For example, what do you call someone who professionally creates software? One analysis shows that the most popular job title associated with this function has evolved from Computer Programmer in the 1980s, to

Read More »
    Your Cart
    Your cart is emptyReturn to Shop
    Scroll to Top