In a previous experiment, AI identified roughly half the usability problems that trained researchers found in a video of a usability test session. That sounds promising. If AI can find usability issues, it can substantially increase the amount of usability testing that research teams can conduct. But in our analysis of that video, AI generated nearly
While humans are error prone, many of the unintended actions, delays, frustrations and confusions come from interfaces: Pushing instead of pulling a door Not being able to set the time on a clock Unintentionally voting for the wrong candidate on a ballot Turning on the wrong burner on a stove Not understanding an error message
A lot happens when you observe a user during a usability test. There’s the interface, the utterances, the body language and the metrics. This rich experience can result in a long list of usability issues. Issues can range from cosmetic (a user finds the type in ALL CAPS a bit much) to catastrophic (data loss),
If you ask independent usability evaluators to run a usability test and report the problems found you’ll get largely different lists of problems. While there are many causes for the differences, one major reason is that evaluators disagree on what constitutes a problem. Usability is often at odds with security and business interests—what’s best for
Let’s imagine you are testing five users as part of an iterative testing services to find and fix problems. During the test only one user encounters a problem with logging in. To fix this particular problem would take a lot of effort and the small sample size is met with skepticism from the overburdened and
Nielsen derives his “five users is enough” formula from a paper he and Tom Landauer published in 1993. Before Nielsen and Landauer James Lewis of IBM proposed a very similar problem detection formula in 1982 based on the binomial probability formula.[4] Lewis stated that: The binomial probability theorem can be used to determine the probability
