MeasuringU

Seven Reasons Survey Questions Are Answered Incorrectly

By Jeff Sauro, PhD and Jim Lewis, PhD . April 6, 2021 . Tags: Questionnaires, Rating Scales, Rating Scale, Sample Size

Surveys are an essential method for collecting data. But like all research methods, surveys have their limitations. Unless the survey is administered by a facilitator, a respondent has only the survey's instructions, questions, and response options for guidance. Earlier we wrote about ways to improve the clarity of your questions and certain words to watch for when writing questions. But, despite your best efforts, inevitably

Seven Reasons People Misinterpret Survey Questions

By Jeff Sauro, PhD and Jim Lewis, PhD . March 23, 2021 . Tags: Rating Scales, Satisfaction, Rating Scale

Like in all research methods, many things can go wrong in surveys, from problems with sampling to mistakes in analysis. To draw valid conclusions from your survey, you need accurate responses. But participants may provide inaccurate information. They could forget the answers to questions or just answer questions incorrectly. One common reason respondents answer survey questions incorrectly is that they misinterpret the question. Understanding the

Nine Words to Watch for When Writing Survey Questions

By Jeff Sauro, PhD and Jim Lewis, PhD . February 23, 2021 . Tags: Rating Scales, Rating Scale, Surveys, Survey

In UX research, both studies and surveys contain a lot of questions. Getting those questions right can go a long way in improving the clarity and quality of the findings. For example, we’ve recently written about how to make survey questions clearer. And while there are many stories of how the change of a single word in a survey question can lead to different results

UX and Net Promoter Benchmarks of Auto Insurance Websites

By Jim Lewis, PhD and Jeff Sauro, PhD . January 26, 2021 . Tags: Report, SUPR-Q, Net Promoter Score, Rating Scales, Key Driver, Benchmarking, Sample Size

Fifteen minutes could save you 15%. Nationwide is on your side. You’re in good hands with Allstate. Auto insurance commercials are ubiquitous. It’s no wonder, considering the market. In 2020, the population of the United States was 331,000,000, and 230,000,000 Americans were licensed drivers. If you drive, you should have auto insurance. Until recently, most customer interactions with an auto insurance company were through its

Five Reasons to Use Open-Ended Questions

By Jeff Sauro, PhD and Jim Lewis, PhD . January 12, 2021 . Tags: Single Ease Question, Net Promoter Score, Rating Scales, SEQ, NPS

Despite the ease with which you can create surveys using software like our MUIQ platform, selecting specific questions and response options can be a bit more involved. Most surveys contain a mix of closed-ended (often rating scales) and open-ended questions. We’ve previously discussed 15 types of common rating scales and have published numerous articles in which we investigated their measurement properties. Now we turn our

Revisiting the Evidence for the Left-Side Bias in Rating Scales

By Jim Lewis, PhD and Jeff Sauro, PhD . October 20, 2020 . Tags: Bias, TAM, Questionnaires, Rating Scales

Are people more likely to select response options that are on the left side of a rating scale? About ten years ago, we provided a brief literature review of the published evidence, which suggested that this so-called left-side bias not only existed but also was detected almost 100 years ago in some of the earliest rating scales. Across the publications we reviewed, the effect size

The UX of Vacation Rental Websites

By Jeff Sauro, PhD and Jim Lewis, PhD . October 6, 2020 . Tags: SUPR-Q, Net Promoter Score, Rating Scales, Sample Size

The COVID-19 pandemic has led to significant changes in how people have vacationed in 2020. To get away from it all without spending time in crowded places, vacationers have turned to vacation rental websites and have planned longer stays. For example, Airbnb recently reported a year-to-year doubling of long-term (>28 days) rentals and a shift from urban to rural stays. These changes make sense given

How to Convert Between Five- and Seven-Point Scales

By Jim Lewis, PhD and Jeff Sauro, PhD . June 16, 2020 . Tags: Rating Scales, Rating Scale, Surveys, Survey

There is plenty of debate about the best way to quantify attitudes and experiences with rating scales. And among those debates, perhaps the most popular question is the “right” number of response options to use for rating scales. For example, is an eleven-point scale too difficult for people to understand? Is a three-point scale insufficient for capturing extreme attitudes? Most research on this topic shows

Confounded Experimental Designs, Part 1: Incomplete Factorial Designs

By Jim Lewis, PhD and Jeff Sauro, PhD . June 2, 2020 . Tags: Variables, Research, Rating Scales, Surveys

UX research and UX measurement can be seen as an extension of experimental design. At the heart of experimental design lie variables. Earlier we wrote about different kinds of variables. In short, dependent variables are what you get (outcomes), independent variables are what you set, and extraneous variables are what you can’t forget (to account for). When you measure a user experience using metrics—for example,

Three Questionnaires for Measuring Voice Interaction Experiences

By Jim Lewis, PhD and Jeff Sauro, PhD . March 3, 2020 . Tags: Voice Interaction, Rating Scales, Customer Experience, Questionnaire

Human: Computer, can you recognize speech? Computer: I think you said, can you wreck a nice beach? Both the quality of synthesized speech and the capability of communicating with a computer using your voice have come a long way since the debut of this technology in the 1970s. One of the most famous synthetic voices was the one used by Stephen Hawking. Some of its

Rating Scales

by Jeff Sauro, PhD and Jim Lewis, PhD | April 6, 2021

Browse Content by Topic

Seven Reasons Survey Questions Are Answered Incorrectly

By Jeff Sauro, PhD and Jim Lewis, PhD . April 6, 2021 . Tags: Questionnaires, Rating Scales, Rating Scale, Sample Size

Seven Reasons People Misinterpret Survey Questions

By Jeff Sauro, PhD and Jim Lewis, PhD . March 23, 2021 . Tags: Rating Scales, Satisfaction, Rating Scale

Nine Words to Watch for When Writing Survey Questions

By Jeff Sauro, PhD and Jim Lewis, PhD . February 23, 2021 . Tags: Rating Scales, Rating Scale, Surveys, Survey

UX and Net Promoter Benchmarks of Auto Insurance Websites

By Jim Lewis, PhD and Jeff Sauro, PhD . January 26, 2021 . Tags: Report, SUPR-Q, Net Promoter Score, Rating Scales, Key Driver, Benchmarking, Sample Size

Five Reasons to Use Open-Ended Questions

By Jeff Sauro, PhD and Jim Lewis, PhD . January 12, 2021 . Tags: Single Ease Question, Net Promoter Score, Rating Scales, SEQ, NPS

Revisiting the Evidence for the Left-Side Bias in Rating Scales

By Jim Lewis, PhD and Jeff Sauro, PhD . October 20, 2020 . Tags: Bias, TAM, Questionnaires, Rating Scales

The UX of Vacation Rental Websites

By Jeff Sauro, PhD and Jim Lewis, PhD . October 6, 2020 . Tags: SUPR-Q, Net Promoter Score, Rating Scales, Sample Size

How to Convert Between Five- and Seven-Point Scales

By Jim Lewis, PhD and Jeff Sauro, PhD . June 16, 2020 . Tags: Rating Scales, Rating Scale, Surveys, Survey

Confounded Experimental Designs, Part 1: Incomplete Factorial Designs

By Jim Lewis, PhD and Jeff Sauro, PhD . June 2, 2020 . Tags: Variables, Research, Rating Scales, Surveys

Three Questionnaires for Measuring Voice Interaction Experiences

By Jim Lewis, PhD and Jeff Sauro, PhD . March 3, 2020 . Tags: Voice Interaction, Rating Scales, Customer Experience, Questionnaire