What should you do when your sample doesn’t match the known population composition on key variables like prior experience? One approach is to weight your data to rebalance the sample. In a previous article, we discussed how to weight means (such as from rating scales) when there are differences between group proportions in a sample
In a previous article, we discussed the pros and cons of using weights to compensate for differences between a sample and a reference population. Due to its risks, the consensus about weighting is that it’s a method of last resort when (1) it’s critically important for proportions of sample groups to match a reference population
To weight, or not to weight, that is the question: Whether ’tis nobler in the mind to suffer Discrepancies between sample and population proportions, Or to take arms against the gaps But by weighting them, to distort the truth… Weighting plays an important role in how we measure, and even what we consider, the truth.
Sample size computations can seem like an art. Some assumptions are involved when computing sample sizes, but it should be more math than magic. A key ingredient needed to cook up a sample size estimate is the standard deviation. You need yeast to make bread, and you need a measure of variability to make an
In an earlier article, we examined a large dataset of rating scale data. After analyzing over 100,000 individual responses from 4,048 multipoint items across 25 studies, we reported the typical standard deviations for five-, seven-, and eleven-point items. We found that the average standard deviation tended to be around 25% of the maximum range of
What is the impact if you sample a lot of your population in a survey? Many statistical calculations—for example, confidence intervals, statistical comparisons (e.g., the two-sample t-test), and their sample size estimates—assume that your sample is a tiny fraction of your population. But what if you have a relatively modest population size (e.g., IT decision-makers
Proportional data is common in both UX research and the larger scientific literature. You can use proportions to help make data-driven decisions just about anywhere: Which design converts more? Which product is preferred? Does the new interface have a higher completion rate? What proportion of users had a problem with registering? Consequently, you’ll likely want to
We use the seven-point Single Ease Question (SEQ®) frequently in our practice, as do many other UX researchers. One reason for its popularity is the body of research that started in the mid-2000s with the comparison of the SEQ to other similar short measures of perceived ease-of-use, the generation of a normative SEQ database, and
If you took an intro to stats class (or if you know just enough to be dangerous), you probably recall two things: something about Mark Twain’s “lies, damned lies …,” and that your data needs to be normally distributed. Turns out both are only partly true. Mark Twain did write the famous quote, but he
Microsoft Word is a widely used word processing program, part of the Microsoft Office suite of programs. While its dominance has been challenged recently by Google Docs, Word still leads on the features list, providing many features that Google’s offering lacks. But adding features can also add to bloat, making common tasks harder as users
