UX researchers use a lot of rating scales. We recommend using standardized rating scales when possible. One of the benefits of some standardized scales, such as the SUS, SUPR-Q®, and UX-Lite®, is that you have a reference database of historical data. But there’s not always a standardized questionnaire for everything you’re hoping to measure, so
AI can “watch” videos. It can even generate a list of problems. In some cases, these problem lists seem to be reasonably consistent (reliable). But consistency is not accuracy. Are these real problems or just sophisticated AI slop generated consistently by autocorrect for video? How can we know? One way to find out is to
It looks like AI can “watch” videos. And if AI can watch videos, it can likely extract UI problems. That suggests it has the potential to support UX research. So maybe AI can “watch” a video and detect some problems. But if you run the same video through AI multiple times, do you get the
You may have become numb to the overhyped headlines about AI. But it’d be wrong to dismiss the impact AI can have on our industry, not only because of job displacement, but also of helping us do our jobs more effectively (hopefully). To separate the hype and hysteria, we at MeasuringU think about AI’s impact
One of the hardest parts of conducting user and market research is recruiting participants. It takes time, costs money, and on top of that, there are no-shows and fraudsters. Now imagine being able to conduct UX research without the hassle of recruiting the “U.” Enter the idea of AI-generated synthetic users that offer the promise
We’ve written a lot about confidence intervals for the last two decades. We especially encourage them for small sample studies. Some of you even bought into our recommendation and use them yourselves (a decision we continue to support). But maybe you’ve heard about Bayesian credible intervals and wonder if you should be using them instead.
Surveys are a common way to measure attitudes, behaviors, and intentions related to products and services. But large surveys can include dozens of questions and multiple demographic segments, which can mean hundreds of potential comparisons. How do you present all those results in a way stakeholders can quickly scan? You can use a slide deck
Say you collect rating scale data from dozens of users across ten apps. To analyze the data, you compute medians because you learned that rating scale data isn’t interval or ratio data. The medians of all ten apps end up the same. They’re all 4! If you rely on the medians, you’d conclude the apps
“That design will never work.” You may have had that thought before you even ran your first participant in a usability test. If you’ve seen enough users struggle and conducted enough usability tests, then you probably have some idea about how well or poorly a task attempt may go for prototypes or even commercially available
The completion rate jumped from 20% to 80%. That’s a large effect size. If it had gone from 20% to 21%? Much smaller effect. It’s easy to get caught up in the mechanics of significance testing and p-values. But even before those tools existed, researchers were measuring effect sizes. Effect sizes remain fundamental to understanding
