When we wrote Quantifying the User Experience, we put confidence intervals before tests of statistical significance. We generally find fluency with confidence intervals to be easier to achieve and of more value than with formal hypothesis testing. We also teach confidence intervals in our workshops on statistical methods. Most people, even non-researchers, have been exposed
Despite the ease with which you can create surveys using software like our MUIQ platform, selecting specific questions and response options can be a bit more involved. Most surveys contain a mix of closed-ended (often rating scales) and open-ended questions. We’ve previously discussed 15 types of common rating scales and have published numerous articles in
Sliders are a type of visual analog scale that can be used with many online survey tools such as our MUIQ platform. The literature on their overall effectiveness is mixed (Roster et al., 2015). On the positive side, evidence indicates that sliders might be more engaging to respondents. On the negative side, evidence also indicates
During the fall in the northern hemisphere, leaves change colors, birds fly south, and the temperature gets colder. Do the birds change the color of the leaves, and does their departure make the temperature colder? What if you gave participants two versions of a rating scale, with the first having responses ordered from strongly disagree
There are a lot of opinions about the best formats for agreement scales. Sometimes those opinions are strongly held and can lead to lengthy, heated discussions within research teams. When format differences affect measurement properties, those discussions may be time well spent, but when the formats don’t matter (or matter very little), the time is
Somewhat agree, very satisfied, extremely likely. The labels used on the points of rating scales can affect responses in often unpredictable ways. What’s more, certain terms can get lost in translation when writing surveys for international usage. Some terms may have subtly different meanings, possibly making cross-cultural comparisons problematic. While numbers are universally understood and
The Mean Opinion Scale (MOS) is a standardized questionnaire used to assess synthetic speech. The quality of synthetic speech strongly affects the user experience of working with conversational systems, with listeners making rapid and unconscious judgments of the speaker’s personality, so it’s important to have standardized methods for its assessment. In an earlier article, we
The quality of the electronic transmission of the human voice has come a long way since Bell summoned Watson. But even with all the advancement in technology, “Can you hear me now?” is still part of our modern lexicon. Voice—both human and digital—plays an increasingly important role in interactions with our devices. But before you
Five-star reviews. Whether you’re rating a product on Amazon, a dining experience on Yelp, or a mobile app in the App or Play Store, you can see that the five-star rating system is quite ubiquitous. Does the familiarity of stars offer a better rating system than traditional numbered scales? We recently reported a comparison between standard
There are many ways to format rating scales. Recently we have explored Labeling neutral points Labeling all or some response options Altering the number of response options Comparing agreement vs. item-specific endpoint labels Each of these formatting decisions has a variety of opinions and research, both pro and con, in the scientific literature at large.
