Learn more about using our MUiQ platform to conduct your own SUPR-Q studies here: MUiQ Testing Platform
The SUPR-Q (Standardized User Experience Percentile Rank Questionnaire) is an 8 item questionnaire for measuring the quality of the website user experience.
In addition to an overall quality metric (similar to satisfaction), the SUPR-Q provides measures of Usability, Credibility/Trust, Loyalty and Appearance. It’s used by over 100 organizations from several industries: from eCommerce and Travel to government services and education. Customers include Walmart, PayPal, Lenovo, Kelley Blue Book, Indeed and the Canadian Government.
This SUPR-Q license includes a downloadable spreadsheet that allows you to compare your website (one domain) to the various websites in the tool.
The SUPR-Q license term is for 1 year and includes updates once a quarter. The updates are included with the annual license fee.¬† Note, the older license (prior to June 2017) did not include updates.¬† Each quarter we update the SUPR-Q database with a new set of websites, usually between 5 and 20 new websites and update the scores algorithm accordingly.
SUPR-Q Full License: See the 200+ websites and industries behind the comparison, and sort by industry and by sub-scale (trust, usability, loyalty, and appearance).
Individual License: provides one person within an organization access to the license, to compare one domain to over 200 websites, see the industries behind the comparison, and sort by industry and sub-scale.
Site License: provides multiple people within the organization access to the license and compare one domain to over 200 websites, see the industries behind the comparison, and sort by industry and sub-scale.
Purchase example:¬† Purchase the individual license, and grant one person (one individual) in the organization access to the license.¬† This person can compare a website (one domain) to over 200 websites, access the industries behind the comparison, and sort by industry and sub-scale.
Contact us for educational rates and additional questions about using it for multiple domains.
How it Works
Use the SUPR-Q as part of a usability test, or in a site intercept as an ongoing assessment tool. For a usability test, have your users answer the 8 questions after they’ve completed realistic task scenarios. Your scored results can then be compared to hundreds of other websites.
- Have users of a website answer the 8 SUPR-Q items: Do this either using a pop-up on a website, a survey or after a usability test. Note: The older version of the SUPR-Q used 13 items.
- Paste the responses into the calculator: The calculator automatically generates percentile ranks for a global score and each of the four sub-scales (Usability, Credibility/Trust, Loyalty and Appearance).
- Compare your results to other websites: With the Full License you can see how your score compares a list of 200 websites across twenty industries, government, and academic institutions. The Limited License offers the comparative score at a discounted price.
- Demonstration of the SUPR-Q Calculator
The SUPR-Q (Standardized User Experience Percentile Rank Questionnaire) measures the essential aspects of a website in just 8 multiple choice questions.
It was developed through a process called psychometric qualification, in which the best 8 items from a pool of 75 candidate items were identified. A peer-reviewed journal article discusses the validation process in detail. The items were winnowed down by only retaining those which generated high internal reliability, loaded highest across common factors, had the highest item-total correlation, and could discriminate best between websites.
The database contains responses from more than 10,000 participants and 150 websites. The websites are updated on an ongoing basis (about once per quarter).
Usability: If users can’t accomplish what they want to do, find the product they’re looking for, or complete their purchase, it’s as if the information or product doesn’t exist. A usable experience means a profitable experience.
Credibility (Trust, Value & Comfort): Does the website sell products and collect credit card information? Are you gathering email addresses to build a subscriber base? If users don’t trust your website, which for many companies is synonymous with their company and brand, they won’t give up their information and website growth is impeded.
Loyalty: Are users talking about your website favorably or are they telling their friends to avoid it like the plague? Will they return to the website and purchase more, or at least see what you have to say? The two questions asked under Loyalty touch upon repeat usage from existing customers and net-new usage from new customers.
Appearance: Is your website looking like it’s circa 1998, or is the appearance hindering the experience? Users form impressions of your website based on the appearance in just a few seconds.
SUPR-Q General Information
How usable is my website, what should I fix?
It’s important to have a mix of both qualitative comments from your users and quantitative benchmarks to know how much effort you should invest in improving your website. Interpreting open ended feedback is an important step in identifying what to improve on your website. One problem with open-ended comments is that they can be difficult to put in context.
A SUPR-Q score tells you if visitors think your website is usable. If your SUPR-Q score is low compared to your peers, then start fixing the things identified in your comments and consider commissioning a larger usability study to generate more substantial design improvements. If your SUPR-Q score is high, then at least you know you’re doing something right! You should constantly be updating your website based on feedback, but high SUPR-Q scores suggest major usability improvements and would be a lower priority.
The SUPR-Q is a rating scale to measure perceptions of Usability, Credibility/Trust, Loyalty, and Appearance for websites.
How usable is your website? Will your visitors think the website is credible and trustworthy? While it is important to continuously make improvements in your user experience, it’s good to know how well your website stacks up against other websites on the important aspects of usability, credibility/trust, loyalty, and appearance. Are users able to find what they’re looking for? Do they feel comfortable providing personal information? Will they return and become loyal customers?
What are the SUPR-Q items?
- This website is easy to use.
- It is easy to navigate within the website.
Credibility: (Trust, Value & Comfort)
- The information on the website is credible.
- The information on the website is trustworthy.
- I feel comfortable purchasing from this website. (Alternate for eCommerce websites)
- I feel confident conducting business with this website. (Alternate eCommerce websites)
- How likely are you to recommend this website to a friend or colleague?
- I will likely visit this website in the future.
- I found the website to be attractive.
- The website has a clean and simple presentation.
You are welcome to use the SUPR-Q questions without purchasing the SUPR-Q license. Please simply cite this reference (Sauro, 2015)
What are the response options?
There are five response options for 7 of the questions. You can randomize the order of the responses as long as you keep track of the item number when scoring the responses. Amazon is used as an example website; substitute the name of your website.
The final question “How likely are you to recommend this website to a friend or colleague?” has 11 response options and can be displayed like the example below. It needs to start with a 0 and end with a 10, and have both end-points and the middle option labeled. It also provides the commonly used Net Promoter Score.
How do you score the responses?
Scoring is simple. Halve the score for the Likelihood to Recommend question and then average all eight scores (the seven SUPR-Q scores and the halved Likelihood to Recommend score). This is your SUPR-Q score. This can then be compared to the industry benchmarks.
How is the SUPR-Q score generated?
The SUPR-Q score is expressed as a percentile (hence percentile rank), so a SUPR-Q score of 75% would mean the global score is higher than 75% of all websites tested in the database. The same percentile ranking procedure is then done for all of the four factors and by each of the 8 questions. This can also be done within a specific industry. It is common to have a range of rankings for each of the four subscales. Some websites might score high in usability but low with respect to loyalty.
What websites make-up the SUPR-Q database?
Data from 200 websites make up the SUPR-Q database. Most websites are US based and were tested on users from North America. Roughly 10% of the users come from countries outside of the US & Canada including: UK, Australia, Romania, India, Japan, Mexico and Hong Kong. Examples include large and small ecommerce websites, mobile phone carriers, travel websites and governments.
Is the SUPR-Q reliable?
The SUPR-Q has been used on over 200 websites and by over 4500 users from 13 countries (90% US & Canada). As part of the validation of the instrument, the internal reliability was measured using Cronbach Alpha. The global score and each factor exhibits strong internal reliability as can be seen in Figure 1 below.
Is the SUPR-Q Valid?
To ensure the SUPR-Q provides a valid measure of usability, credibility, trust and appearance it was compared to other known instruments which measure similar constructs. The SUPR-Q global score and usability score have a very strong correlation with the System Usability Scale (SUS) r > .88 p <.01. Additionally, 108 users across 8 different websites completed both the SUPR-Q and the WAMMI questionnaire. The usability factor and global SUPR-Q score also have strong correlations (r > .88, p <.01) with the average responses from the 20 WAMMI questions. A combination of just two questions from the SUPR-Q account for 88% of the variability in the 20 WAMMI questions. The SUPR-Q was able to discriminate well against websites which have been known to have poor usability, trust, loyalty and appearance against others which score highly on these attributes.
How is SUPR-Q like SUS?
The System Usability Scale (SUS) is a popular 10 item questionnaire which measures the perceptions of usability. The SUS was developed over 20 years ago and provides a reliable estimate of usability. Its questions are generic enough to use on a wide variety of software and hardware applications. However, as a result of this generic quality, some questions don’t fit well with websites (e.g. needing the support of a technical person). The result is that while SUS still provides a reliable picture of users’ attitudes toward usability, it is limited to only perceptions of usability and excludes other important factors such as credibility/trust, loyalty, and appearance.
The SUPR-Q overall score has a strong correlation with a SUS score, r = .87, p < .001, meaning just four questions account for 87 percent of the variation in SUS (.87 squared). These questions are a good substitute for SUS on websites.
How is the SUPR-Q different than other standardized questionnaires?
A good standardized questionnaire provides both a valid and reliable instrument as well as a relative ranking based on a dataset. The SUS is a good instrument but a raw SUS score is difficult to interpret without knowing how other websites score (e.g. is a 70 good?). Questionnaires like the WAMMI and SUPR-Q both provide relative rankings expressed as percentages. Therefore a percentile score of 50 is an average score, or roughly half the websites have received better scores and half received worse scores. One problem with the WAMMI is that you don’t know what websites make up the scoring system. In any large dataset there will be some websites that are good comparisons and others which are irrelevant. What makes SUPR-Q different is that you know what websites are in the dataset. What’s more, you can see how well your SUPR-Q scores compares to a subset of up to 10 other websites or compared to the industry.