MeasuringU: All Articles

Evaluating NPS Confidence Intervals with Real-World Data

By Jim Lewis, PhD and Jeff Sauro, PhD . January 19, 2021 . Tags: Net Promoter Score, Rating Scale, NPS, UX, Sample Size, User Experience

The Net Promoter Score (NPS) is a popular business metric used to track customer loyalty. It uses a single likelihood-to-recommend (LTR) question (“How likely is it that you will recommend our company to a friend or colleague?”) with 11 scale steps from 0 (Not at all likely) to 10 (Extremely likely). In NPS terminology, respondents who select 9 or 10 on the LTR question are

Five Reasons to Use Open-Ended Questions

By Jeff Sauro, PhD and Jim Lewis, PhD . January 12, 2021 . Tags: Single Ease Question, Net Promoter Score, Rating Scales, SEQ, NPS

Despite the ease with which you can create surveys using software like our MUIQ platform, selecting specific questions and response options can be a bit more involved. Most surveys contain a mix of closed-ended (often rating scales) and open-ended questions. We’ve previously discussed 15 types of common rating scales and have published numerous articles in which we investigated their measurement properties. Now we turn our

Confidence Intervals for Net Promoter Scores

By Jim Lewis, PhD and Jeff Sauro, PhD . January 5, 2021 . Tags: Net Promoter Score, Confidence, Confidence Intervals, NPS

The Net Promoter Score (NPS) is a widely used metric, but it can be tricky to work with statistically. One of the first statistical steps we recommend that researchers take is to add confidence intervals around their metrics. Confidence intervals provide a good visualization of how precise estimates from samples are. They are particularly helpful in longitudinal research to help differentiate the inevitable fluctuations in

48 UX Metrics, Methods, & Measurement Articles from 2020

By Jim Lewis, PhD and Jeff Sauro, PhD . December 29, 2020 . Tags: UMUX-lite, Rating Scale, Sample Size, Survey

Happy New Year from all of us at MeasuringU! 2020 was a crazy year, but we still managed to post 48 new articles and continued improving MUIQ, our UX testing platform. We hosted our seventh UX Measurement Bootcamp, this time virtually. The change of format was a challenge, but it was fantastic to work with attendees from all over the world. The topics we wrote

From Functionality to Features: Making the UMUX-Lite Even Simpler

By Jim Lewis, PhD and Jeff Sauro, PhD . December 15, 2020 . Tags: UMUX-lite, Research, Rating Scale

Like pictures and pixels on a screen, words are a type of user interface. Complex language, like complex software, can lead to misunderstanding, so words should communicate effectively while being easy to understand. The solution, to paraphrase William Zinsser, is to use words that are simple and concise—a guideline that also applies to UX questionnaires. This brings us to the UMUX-Lite, an increasingly popular UX

What a Randomization Test Is and How to Run One in R

By Jim Lewis, PhD and Jeff Sauro, PhD . December 8, 2020 . Tags: R, t-test, Randomization, Rating Scale, Sample Size, Statistics

The two-sample t-test is one of the most widely used statistical tests, assessing whether mean differences between two samples are statistically significant. It can be used to compare two samples of many UX metrics, such as SUS scores, SEQ scores, and task times. The t-test, like most statistical tests, has certain requirements (assumptions) for its use. While it’s easy to conduct a two-sample t-test using

From Soared to Plummeted: Can We Quantify Change Verbs?

By Jeff Sauro, PhD and Jim Lewis, PhD . December 1, 2020 . Tags: sliders, UMUX-lite, Questionnaires, Rating Scale, Sample Size

Cases spike, home prices surge, and stock prices tank: we read headlines like these daily. But what is a spike and how much is a surge? When does something crater versus tank or just fall? Headlines are meant to grab our attention. They often communicate the dramatic story the author wants to tell rather than what the data say. It isn’t easy to write headlines.

Rating Scale Best Practices: 8 Topics Examined

By Jeff Sauro, PhD and Jim Lewis, PhD . November 17, 2020 . Tags: Sensitivity, Emoji scale, Star Scale, sliders, LTR, UMUX-lite, Sample Size

Rating scales have been around for close to a century. It’s no wonder there are many questions about best practices and pitfalls to avoid. And like any topic that’s been around for that long, there are urban legends, partial truths, context-dependent findings, and just plain misconceptions about the “right” and “wrong” way to use and interpret rating scales. We’ve been researching and conducting our own

Are Sliders More Sensitive than Numeric Rating Scales?

By Jeff Sauro, PhD and Jim Lewis, PhD . November 10, 2020 . Tags: Sensitivity, slider, sliders, Reliability, Sample Size

Sliders are a type of visual analog scale that can be used with many online survey tools such as our MUIQ platform. The literature on their overall effectiveness is mixed (Roster et al., 2015). On the positive side, evidence indicates that sliders might be more engaging to respondents. On the negative side, evidence also indicates that sliders can be more cognitively and physically challenging than

Latin and Greco-Latin Experimental Designs for UX Research

By Jim Lewis, PhD and Jeff Sauro, PhD . November 3, 2020 . Tags: Latin Squares, Greco-Latin Squares, Research design, Questionnaires, Survey

During the fall in the northern hemisphere, leaves change colors, birds fly south, and the temperature gets colder. Do the birds change the color of the leaves, and does their departure make the temperature colder? What if you gave participants two versions of a rating scale, with the first having responses ordered from strongly disagree to strongly agree and the second reversing the order, and