Having a representative sample is ideal when making inferences about your customer or user population. In practice, it can be difficult to recruit the right proportion of respondents, leaving your sample out of balance with the population. One way to adjust for being off balance is to weight the data you collected to get the
What should you do when your sample doesn’t match the known population composition on key variables like prior experience? One approach is to weight your data to rebalance the sample. In a previous article, we discussed how to weight means (such as from rating scales) when there are differences between group proportions in a sample
In a previous article, we discussed the pros and cons of using weights to compensate for differences between a sample and a reference population. Due to its risks, the consensus about weighting is that it’s a method of last resort when (1) it’s critically important for proportions of sample groups to match a reference population
To weight, or not to weight, that is the question: Whether ’tis nobler in the mind to suffer Discrepancies between sample and population proportions, Or to take arms against the gaps But by weighting them, to distort the truth… Weighting plays an important role in how we measure, and even what we consider, the truth.
