Paraphrasing the statistician George Box, all models are wrong, some are useful, and some can be improved. In a recent article, we reviewed the most common way of modeling problem discovery, which is based on a straightforward application of the cumulative binomial probability formula: P(x≥1) = 1 – (1-p)n. Well, it’s straightforward if you like
Finding and fixing problems in an interface is one of the fundamental priorities of a formative usability test. But how many users should you test with? And how many usability problems are there to be uncovered? These questions have been discussed and debated for decades. Early work on problem discovery suggested that the first few
The fundamental goal of usability testing is to produce highly usable products and services. That’s an uncontroversial statement. Where things can get a bit confusing is how different approaches to usability testing have different ways of achieving that goal. In earlier articles we have described the different types of usability tests but many types still
Usability testing is expensive. At least that has been the perception. But the idea that usability is a nice-to-have ideal that only big companies such as IBM or Microsoft can afford has fortunately evolved. While technology has improved and gotten cheaper, it’s the technique that’s become more accessible and accepted. The discount-usability movement helped emphasize the effectiveness
Usability tests don’t have to be expensive or require a lot of technology. The real value is not in the equipment or technology but in the technique. Usability testing is not a focus group. Nor is usability testing a product demo. You shouldn’t lead participants through a product as if it were a demo and
You don’t need a dedicated usability lab to conduct a usability test. But if you or your organization conducts more than the occasional usability test, which it probably should (another topic in itself), you may want to consider setting up a dedicated usability lab. Having a dedicated space for testing is a hallmark of organizations
Uncovering usability problems is at the heart of usability testing. Problems and insights can be uncovered from observing participants live in a usability lab, using heuristic evaluations, or watching videos of participants. But if you change the person looking for the problems (the evaluator), do you get a different set of problems? The Evaluator Effect
Finding and fixing problems encountered by participants through usability testing generally leads to a better user experience. But not all participants are created equal. One of the major differentiating characteristics is prior experience. People with more experience tend to perform more tasks successfully, more quickly and generally have a more positive attitude about the experience than inexperienced
While UX research may be a priority for you, it probably isn’t for your participants. And participants are a pretty important ingredient in usability testing. If people were predictable, reliable, and always did what they said, few of us would make a living in improving the user experience! Unfortunately, people don’t always show up when
Many researchers are familiar with the Hawthorne Effect in which people act differently when observed. It was named when researchers found workers at the Hawthorne Works factory performed better not because of increased lighting but because they were being watched. This observer effect happens not only with people but also with particles. In physics, the
