Confidence intervals are your frenemies. They are one of the most useful statistical techniques you can apply to customer data. At the same time they can be perplexing and cumbersome. But confidence intervals provide an essential understanding of how much faith we can have in our sample estimates, from any sample size, from 2 to
Which product is the most usable? One of the primary goals of a comparative study is to understand which product or website performs the best or worst on usability metrics such as completion rates or perceptions of usability. Comparisons can be made between competitive products or alternate design concepts. When conducting a comparative usability study,
How concerned should you be with missing responses in your survey? One of the primary concerns with sampling in general is the issue of representativeness. That is, we don’t want to sample only happy customers or those who come from large companies instead of small companies if we’re trying to make the right decisions about
It’s that time of year again: March Madness. The Madness in March comes from the NCAA College basketball tournament, with unanticipated winners and losers with dozens of games packed into the final days of March. It’s also the time of year where a lot of people start working directly with probability, whether they know it
Most people are comfortable with the concept of an average or percentage as a measure of quality. An equally important component of measuring the user experience is to understand variability. Here are 10 things to know about measuring variability in the user experience. Variability is inherent to measuring human performance. People have different browsing patterns,
The difference is statistically significant. When using statistics to make comparisons between designs, it’s not enough to just say differences are statistically significant or only report the p-values. With large sample sizes in surveys, unmoderated usability testing, or A/B testing you are likely to find statistical significance with your comparisons. What you need to know
It’s often said that you can get statistics to show anything you want. And while this is true to an extent, it’s equally true of any method, qualitative or quantitative. With statistics, however, it’s a bit harder because there is a numeric audit trail that allows others to better understand your assumptions and conclusions. Unfortunately
Some people think that if you have a small sample size you can’t use statistics. Put simply, this is wrong, but it’s a common misconception. There are appropriate statistical methods to deal with small sample sizes. Although one researcher’s “small” is another’s large, when I refer to small sample sizes I mean studies that have
Usability is hardly physics or chemistry. But there are some important principles from decades of research in Human Computer Interaction (HCI) that apply to design and user research. Here are five famous laws that can be applied to improving the user experience of applications and websites: Miller’s Law of Short Term Memory Load: The psychologist
The science project. It’s something we all have completed and if you have kids might see each year at the school science fair. Does an expensive baseball travel farther than a cheaper one? Which melts an ice block quicker, salt water or tap water? Does changing the amount of vinegar affect the color when dying
