“Don’t include numbers when reporting the results of small-sample research studies!” “If you must, definitely don’t use percentages!” “And of course, don’t even think about using statistics!” We regularly hear variations of this advice from well-intentioned researchers, often senior ones. In 2005, we encountered this debate among UX professionals when we participated in a workshop
Proportional data is common in both UX research and the larger scientific literature. You can use proportions to help make data-driven decisions just about anywhere: Which design converts more? Which product is preferred? Does the new interface have a higher completion rate? What proportion of users had a problem with registering? Consequently, you’ll likely want to
The two-sample t-test is one of the most widely used statistical tests, assessing whether mean differences between two samples are statistically significant. It can be used to compare two samples of many UX metrics, such as SUS scores, SEQ scores, and task times. The t-test, like most statistical tests, has certain requirements (assumptions) for its
Understanding who your users are and what they think about an experience is an essential step for measuring and improving the user experience. Part of understanding your users is understanding how they are similar and different with respect to demographics, psychographics, and behaviors. These groupings are often called clusters or segments to refer to the
Researchers rely heavily on sampling. It’s rarely possible, or even makes sense, to measure every single person of a population (all customers, all prospects, all homeowners, etc.). But when you use a sample, the average value you observe (e.g. a completion rate, average satisfaction) differs from the actual population average. Consequently, the differences between designs
You can’t see customer satisfaction. You can’t see usability. There isn’t a thermometer that directly measures someone’s intelligence. While we can talk about satisfied customers, usable products, or smart people, there isn’t a direct way to measure these abstract concepts. And clearly these concepts vary. We’ve all had experiences that left us feeling unsatisfied or
There are a number of variables that affect UX metrics. In most cases though, you’ll simply want to measure the user experience and not these other “nuisance variables” that may mask the experience users have with an interface. This is especially the case when making comparisons. In a comparative analysis you use multiple measures to
Statistics can be daunting, especially for UX professionals who aren’t particularly excited about the idea of using numbers to improve designs. But like any skill that can be learned, it takes some time to understand statistical concepts and put them into practice. Most participants at our UX Boot Camp go from little knowledge of statistics
Excel is an invaluable tool for analyzing and displaying data. In Part 1 I covered some essentials Excel skills, such as conditionals, absolute references and the fill handle. In this second part I’ll cover a few more advanced functionalities that mimic database manipulations. You can also find these examples in the downloadable spreadsheet. 1. VLOOKUP
Excel is a powerful program. It’s like an onion, peeling back layers to reveal increasingly specialized functions. The minute you think you’ve mastered it, you discover a new set of functions. It can take years to learn it and unfortunately there’s not usually a class on learning Excel in university. Students have to pick things
