{"id":56,"date":"2011-01-26T11:30:00","date_gmt":"2011-01-26T11:30:00","guid":{"rendered":"http:\/\/measuringu.com\/normal-nps\/"},"modified":"2021-08-12T08:41:23","modified_gmt":"2021-08-12T14:41:23","slug":"normal-nps","status":"publish","type":"post","link":"https:\/\/measuringu.com\/normal-nps\/","title":{"rendered":"Are Net Promoter Scores Normally Distributed?"},"content":{"rendered":"
<\/a>Responses to rating-scale data typically don’t follow a normal distribution.<\/p>\n However, this is unlikely to affect the accuracy of statistical calculations because the distribution of error in the measurement is normally distributed.<\/p>\n Top-box scoring of rating-scale data<\/a> can provide an easy way to summarize or segment your data in the absence of a benchmark or comparison test.<\/p>\n Another reason top-box scores are used with rating-scale data like the Net Promoter Score is that there is a concern that the data are not normally distributed and thus make statistical calculations inaccurate.<\/p>\n By reporting on just the frequencies for each response you avoid problems with assumptions about normality. Unfortunately, condensing 11 responses into 2 or 3 sacrifices important information about precision and variability.<\/p>\n There will always be value in segmenting responses into groups for concise reporting (especially to executives). But when you want to determine whether your score has statistically improved, you’ll want to use the mean and standard deviation because they provide more precision at smaller sample sizes. Doing so means that you need to consider the distribution of your data.<\/p>\n Even if you know enough about statistics to be dangerous, you’ve probably heard the warning that you need to be sure your data are normally distributed.<\/p>\n Fortunately, you don’t need to sit through a semester of statistics to understand the role of the normal distribution in analyzing rating-scale data like the question used to compute Net Promoter Scores.<\/p>\n A normal distribution (sometimes called Gaussian just to confuse people) refers to data that, when graphed, “distributes” in a symmetrical bell shape with the bulk of the values falling close to the middle.<\/p>\n Normal distributions can be found everywhere: height, weight and IQ scores form some of the more famous normal distributions.The chart below shows the distribution of the heights of 500 North American men.<\/p>\n You can see the characteristic bell shape. The bulk of values fall close to the average height of 5’10” (178 cm) and roughly the same proportion of men are taller or shorter than average.<\/p>\n The popular Net Promoter Score measures customer loyalty using the following question: “How likely are you to recommend a product to a friend?” with responses on an 11-point rating-scale.<\/p>\n Here is the graph of the 673 responses to the “likelihood to recommend” question for a consumer software product. The mean response is 8.4 with a standard deviation of 1.8.<\/p>\n The graph hardly looks like a bell and certainly isn’t symmetric. It’s no wonder researchers have concerns using common statistical techniques like confidence intervals, t-tests or even the mean and standard deviation. When they see non-normal data like this they run!<\/p>\n Normality is important for two reasons:<\/p>\n By error in measurement I’m not talking about the kind that happens when someone misunderstands a question or miscodes the data from a survey. I’m talking about the unsystematic kind that comes from any sample.<\/p>\n When we calculate the mean from a sample, it estimates the unknown population mean. It is almost surely off\u2014over or under\u2014by some amount.<\/p>\n The difference between our sample mean and population mean is called sampling error<\/span>, and it forms its own distribution. We want this distribution to be normal. If our sample of data is normal, then the distribution of sample means (the sampling error) is also normal.<\/p>\n Unfortunately, almost all rating scale data is not normal, so we need to examine the distribution of sample means. But how can we know what this distribution of all sample means looks like if we have only one sample mean?<\/p>\n If we had a lot of time on our hands, we could randomly ask 30 people if they’d recommend the product to a friend. We’d find the mean, graph it, and then rinse and repeat a million times. Or we could simulate that exercise by taking a lot of smaller random samples from a larger sample of data and using a few lines of code.<\/p>\n I chose the latter approach.<\/p>\n I took the large sample of 673 responses and wrote a short program which sampled random responses and computed the mean. I did this at sample sizes of 30, 10 and 5 and repeated it 1000 times for each sample size. The graphs of each distribution of sample means are shown below.<\/p>\nWhat It Means To Be Normal<\/h3>\n
\nFigure 1: Distribution of heights of 500 men from North America. The apostrophe: (e.g. 5′) means feet. <\/span><\/p>\nNet Promoter Data Don’t Look Normal<\/h3>\n
\nFigure 2: Distribution of 673 responses to the “Likelihood to Recommend” question for a consumer software product.<\/span><\/p>\nWhy Normality is Important<\/h3>\n
\n
Error in Measurement<\/h3>\n
The Distribution of Sample Means<\/h4>\n