{"id":170,"date":"2013-06-18T22:00:00","date_gmt":"2013-06-18T22:00:00","guid":{"rendered":"http:\/\/measuringu.com\/10-things-sus\/"},"modified":"2022-03-21T18:08:45","modified_gmt":"2022-03-22T00:08:45","slug":"10-things-sus","status":"publish","type":"post","link":"https:\/\/measuringu.com\/10-things-sus\/","title":{"rendered":"10 Things to Know About the System Usability Scale (SUS)"},"content":{"rendered":"
The System Usability Scale (SUS) is a ten-item questionnaire administered to users for measuring the perceived ease of use of software, hardware, cell phones and websites.<\/p>\n
It’s been around for more than a quarter century, and its wide usage has allowed us to study it extensively and write about it in this blog and in the book, A Practical Guide to the System Usability Scale. <\/a><\/p>\n If you are unfamiliar with the SUS, see the earlier blog<\/a> for some background and fundamentals. Here are 10 things to know when using the SUS:<\/p>\n <\/p>\n Depending on the type of system being tested and its maturity, measures of learnability may be just as important as measures of usability.<\/li>\n What we did see, unfortunately, was a side-effect of alternating. Eleven percent of researchers mis-scored the SUS, because they forgot to reverse the even items. What’s more, 17% of the studies we examined contained problems with participants forgetting to change their response orders when responding to negative items (users were agreeing to at least 3 positive and negative items). These errors are hard to detect because they still generate valid SUS scores. Despite this shortcomings, it’s OK to use the original SUS, just be sure to double check your item coding and, if possible, have a way to follow up with participants if the scoring looks wrong. To help reduce this problem, the SUS Calculator<\/a> flags suspect responses for you.<\/li>\n <\/li>\n While the normal distribution is the reference distribution used in most of the statistical procedures we recommend, it is the distribution of the sample mean which needs to be normally distributed. The graphs below show what the sample mean looks like for sample sizes ranging from 8 to 30. In all cases, the distribution of the sample mean is bell-shaped and symmetrical and allows us to have accurate confidence intervals and p-values, even at small sample sizes.<\/p>\n\n
\nIn a paper we published at CHI<\/a>[pdf]<\/span> a few years ago, we actually found no difference in response biases between an all-positively worded version of SUS and the original version.<\/p>\n
\nThe figure above shows what 311 SUS scores from a single study look like when graphed in a histogram (similar to a bar graph).<\/span><\/p>\n
\nThe figure above shows 1000 Sample Means taken from the dataset shown above at sample sizes of 8, 20 and 30. These sample means show a symetrical bell shape even at small sample sizes and make use of parametric statistics legitimate and accurate.<\/span><\/li>\n
\nThe figure above shows the difference between the average SUS score and a the mean from a sample size of just 5 repeated 1000 times. In 50% of the samples the SUS score from a sample size of 5 was within 6 points of the true SUS score. Not bad for such a small sample size.
\n<\/span>
\nIn other words, if the actual SUS score was a 74, average SUS scores from five users will fall between 66 and 80 half of the time. Seventy-five percent of the time, the score differed by 10 points and 95% of the time, by about 17 points.\u00a0 In other words, you get within the ballpark of the actual SUS score in more than half of the cases with very small sample sizes. For more precise measures of sample sizes, use the SUS Guide and Calculator.<\/a><\/li>\n\n