MeasuringU: All Articles

Nine Words to Watch for When Writing Survey Questions

By Jeff Sauro, PhD and Jim Lewis, PhD . February 23, 2021 . Tags: Rating Scales, Rating Scale, Surveys, Survey

In UX research, both studies and surveys contain a lot of questions. Getting those questions right can go a long way in improving the clarity and quality of the findings. For example, we’ve recently written about how to make survey questions clearer. And while there are many stories of how the change of a single word in a survey question can lead to different results

Evaluating NPS Significance Tests with Real-World Data

By Jim Lewis, PhD and Jeff Sauro, PhD . February 16, 2021 . Tags: Net Promoter Score, Rating Scale, NPS, Sample Size

The Net Promoter Score (NPS) is widely used by organizations. It’s often used to make high-stakes decisions on whether a brand, product, or service has improved or declined. Net Promoter Scores are often tracked on dashboards, and any changes (for better or worse) can have significant consequences: adding or removing features, redirecting budgets, even impacting employee bonuses. Random sampling error, however, can often explain many

Seven Ways to Make Survey Questions Clearer

By Jeff Sauro, PhD and Jim Lewis, PhD . February 9, 2021 . Tags: Questionnaires, Surveys, Survey

The first questionnaires appeared in the mid–18th century (e.g., the “Milles” questionnaire). Scientific surveys have been around for almost a hundred years. Consequently, there are many sources of advice on how to make surveys better. The heart of each survey is the questions asked of respondents. Writing good survey questions involves many of the principles of good writing, in addition to survey-specific guidelines and advice.

How to Statistically Compare Two Net Promoter Scores

By Jim Lewis, PhD and Jeff Sauro, PhD . February 2, 2021 . Tags: Statistical Significance, Net Promoter Score, Confidence, Confidence Intervals, NPS

When we wrote Quantifying the User Experience, we put confidence intervals before tests of statistical significance. We generally find fluency with confidence intervals to be easier to achieve and of more value than with formal hypothesis testing. We also teach confidence intervals in our workshops on statistical methods. Most people, even non-researchers, have been exposed to the concept of margins of error—political polls include them.

UX and Net Promoter Benchmarks of Auto Insurance Websites

By Jim Lewis, PhD and Jeff Sauro, PhD . January 26, 2021 . Tags: Report, SUPR-Q, Net Promoter Score, Rating Scales, Key Driver, Benchmarking, Sample Size

Fifteen minutes could save you 15%. Nationwide is on your side. You’re in good hands with Allstate. Auto insurance commercials are ubiquitous. It’s no wonder, considering the market. In 2020, the population of the United States was 331,000,000, and 230,000,000 Americans were licensed drivers. If you drive, you should have auto insurance. Until recently, most customer interactions with an auto insurance company were through its

Evaluating NPS Confidence Intervals with Real-World Data

By Jim Lewis, PhD and Jeff Sauro, PhD . January 19, 2021 . Tags: Net Promoter Score, Rating Scale, NPS, UX, Sample Size, User Experience

The Net Promoter Score (NPS) is a popular business metric used to track customer loyalty. It uses a single likelihood-to-recommend (LTR) question (“How likely is it that you will recommend our company to a friend or colleague?”) with 11 scale steps from 0 (Not at all likely) to 10 (Extremely likely). In NPS terminology, respondents who select 9 or 10 on the LTR question are

Five Reasons to Use Open-Ended Questions

By Jeff Sauro, PhD and Jim Lewis, PhD . January 12, 2021 . Tags: Single Ease Question, Net Promoter Score, Rating Scales, SEQ, NPS

Despite the ease with which you can create surveys using software like our MUIQ platform, selecting specific questions and response options can be a bit more involved. Most surveys contain a mix of closed-ended (often rating scales) and open-ended questions. We’ve previously discussed 15 types of common rating scales and have published numerous articles in which we investigated their measurement properties. Now we turn our

Confidence Intervals for Net Promoter Scores

By Jim Lewis, PhD and Jeff Sauro, PhD . January 5, 2021 . Tags: Net Promoter Score, Confidence, Confidence Intervals, NPS

The Net Promoter Score (NPS) is a widely used metric, but it can be tricky to work with statistically. One of the first statistical steps we recommend that researchers take is to add confidence intervals around their metrics. Confidence intervals provide a good visualization of how precise estimates from samples are. They are particularly helpful in longitudinal research to help differentiate the inevitable fluctuations in

48 UX Metrics, Methods, & Measurement Articles from 2020

By Jim Lewis, PhD and Jeff Sauro, PhD . December 29, 2020 . Tags: UMUX-lite, Rating Scale, Sample Size, Survey

Happy New Year from all of us at MeasuringU! 2020 was a crazy year, but we still managed to post 48 new articles and continued improving MUIQ, our UX testing platform. We hosted our seventh UX Measurement Bootcamp, this time virtually. The change of format was a challenge, but it was fantastic to work with attendees from all over the world. The topics we wrote

From Functionality to Features: Making the UMUX-Lite Even Simpler

By Jim Lewis, PhD and Jeff Sauro, PhD . December 15, 2020 . Tags: UMUX-lite, Research, Rating Scale

Like pictures and pixels on a screen, words are a type of user interface. Complex language, like complex software, can lead to misunderstanding, so words should communicate effectively while being easy to understand. The solution, to paraphrase William Zinsser, is to use words that are simple and concise—a guideline that also applies to UX questionnaires. This brings us to the UMUX-Lite, an increasingly popular UX