Can You Use The SUS For Websites?

The System Usability Scale (SUS) is the most popular standardized usability questionnaire. SUS was developed about 20 years ago at Digital Equipment Corporation by John Brooke. It’s popular for two reasons: it’s free and short (at only 10 questions). The process of taking a set of ordinary questions and making it into a psychometrically valid and reliable “standardized” questionnaire essentially involves having many users answer a large set of questions against many products and interfaces. You then analyze the responses to see which questions correlate well, tend to a cohesive structure and elicit about the same pattern of responses from people. You throw out questions with low correlations and unusual response patterns to whittle the set down to a manageable set that purports to measure something, in this case perceptions of usability.  This process of standardization was performed on the original super-set of SUS candidate questions some twenty years ago before the web (as we know it) existed.

I’ve seen SUS data for desktop applications, printers, cell-phones, Interactive-Voice Response systems and most recently on websites and web-applications.  Some have argued that the 10 SUS questions just don’t lend themselves well to effectively measuring websites. Websites are typically walk-up-and use customer facing interfaces. Does a SUS score on a website mean the same thing as a SUS score on a word-processor?

1. I think that I would like to use this system frequently.
2. I found the system unnecessarily complex.
3. I thought the system was easy to use.
4. I think that I would need the support of a technical person to be able to use this system.
5. I found the various functions in this system were well integrated.
6. I thought there was too much inconsistency in this system.
7. I would imagine that most people would learn to use this system very quickly.
8. I found the system very cumbersome to use.
9. I felt very confident using the system.
10. I needed to learn a lot of things before I could get going with this system.

Exhibit 1: The 10 SUS questions.

There are certainty some questions which don’t seem well-suited for websites (e.g.Question 4, “I think I’d need the support of a technical person to be able to use this website.”) and I’m sure some of the candidate questions which were thrown out in the original design phase of the questionnaire might be better. So there is good reason to be skeptical about the validity of SUS scores on website. With that said, websites are still interfaces and while some of the questions aren’t the best, they seem to be close enough. Of the data I’ve seen so far, websites with high-perceived usability and applications with high-perceived usability tend to have similar high SUS scores. As I get more data on SUS scores from websites, I’ll be interested to see how well this heavily used psychometric instrument holds up to the changing times.