Topics
Topics
Should a PhD Count as Years of Experience?
You put in the grueling hours. You know what it takes to learn, to persevere, to deliver. There’s nothing quite like the experience of a PhD program. But perhaps a career in academia isn’t what you’re looking for. You decide to leave academia and apply your skills in the UX industry. You look for jobs
Do Statistics Really Require 30 Participants?
Should the sample size n be greater than 30? If you’ve taken any introductory statistics course or an AP statistics class (or helped your child with it), you’ve encountered the n ≥ 30 rule. The “magic number 5” rule we’ve written extensively about applies (with its important caveats) to problem discovery for usability testing. But the
Using the TAC-10 for Screening and Data Cleaning
It’s hard to collect data for UX research, and once you have it, you have to clean it. In a simpler world, all respondents would be honest and focused on providing high-quality information rather than maximizing income, but that’s not the world we live in. From past research, we estimate the prevalence of cheating on
Does AI Find Real UI Problems or Just Hallucinations?
In a previous experiment, AI identified roughly half the usability problems that trained researchers found in a video of a usability test session. That sounds promising. If AI can find usability issues, it can substantially increase the amount of usability testing that research teams can conduct. But in our analysis of that video, AI generated nearly
How Many Years Does It Take to Become a Senior UX Researcher?
What does it take to become a senior UX researcher? An advanced degree? Particular experience and skills, like the number of moderated studies conducted or a variety of methods employed? While all those play a role, the type of job (in-house small-team, in-house large-team, solo researcher, or agency) can affect what you are exposed to.
How to Interpret a Rating Scale Without Historical Data
UX researchers use a lot of rating scales. We recommend using standardized rating scales when possible. One of the benefits of some standardized scales, such as the SUS, SUPR-Q®, and UX-Lite®, is that you have a reference database of historical data. But there’s not always a standardized questionnaire for everything you’re hoping to measure, so
Can AI Detect Usability Problems Like Researchers?
AI can “watch” videos. It can even generate a list of problems. In some cases, these problem lists seem to be reasonably consistent (reliable). But consistency is not accuracy. Are these real problems or just sophisticated AI slop generated consistently by autocorrect for video? How can we know? One way to find out is to
How Reliable Is AI at Finding UI Problems?
It looks like AI can “watch” videos. And if AI can watch videos, it can likely extract UI problems. That suggests it has the potential to support UX research. So maybe AI can “watch” a video and detect some problems. But if you run the same video through AI multiple times, do you get the
Can AI Detect Usability Problems?
You may have become numb to the overhyped headlines about AI. But it’d be wrong to dismiss the impact AI can have on our industry, not only because of job displacement, but also of helping us do our jobs more effectively (hopefully). To separate the hype and hysteria, we at MeasuringU think about AI’s impact
A Review of Experiments with Synthetic Users
One of the hardest parts of conducting user and market research is recruiting participants. It takes time, costs money, and on top of that, there are no-shows and fraudsters. Now imagine being able to conduct UX research without the hassle of recruiting the “U.” Enter the idea of AI-generated synthetic users that offer the promise
Credible vs. Confidence Intervals: Different Meanings but Similar Decisions
We’ve written a lot about confidence intervals for the last two decades. We especially encourage them for small sample studies. Some of you even bought into our recommendation and use them yourselves (a decision we continue to support). But maybe you’ve heard about Bayesian credible intervals and wonder if you should be using them instead.
Bayes’ Law in UX Research:
The Power and Perils of Priors
“That confirms what I expected.” The same data, two different conclusions. A 90% completion rate from 20 participants on a usability test of a checkout flow. Is that completion rate better than the historical average of 78%? One researcher says yes, definitely. Another says no, it’s in line with the historical average. Both are using