There’s a long list of things to fix on any website.
But there are usually a few fixes which will have a disproportionate impact.
To help identify those, we ask users a simple question at the end of our studies: “If you could only fix one thing on this website, what would it be?”
We use an open ended question to collect responses. While this means we get a lot of “nothing” and “I don’t know’s” we also get a lot of valuable insights that are top-of-mind for the users.
We perform a content-analysis and sift through hundreds to thousands of unique responses to group them into larger, more interpretable buckets.
What Users Would Fix
As part of our ongoing data collection effort for the Standardized Universal Percentile Rank (SUPR-Q) we regularly survey users of over 200 websites on a rolling basis. We also ask what one thing each respondent would fix. I looked at a cross selection of 30 surveys we collected in the last year with over 2000 responses to our “what would you fix question.” The 30 websites are shown in the table below.
| Amazon | New York Times |
| Apple | Orbitz |
| BestBuy | PayPal |
| CNN | Shutterfly |
| Craigslist | Snapfish |
| Delta | Target |
| eBay | |
| eTrade | USA.gov |
| Vistaprint | |
| Flickr | Wall Street Journal |
| Foxnews | Walmart |
| GoDaddy | Yahoo |
| IMDB | Yelp |
| YouTube | |
| Netflix | Zappos |
After eliminating the “I don’t knows” and “nothing’s” I put all the things users say they would fix into a word cloud.
Word clouds seem like a vestige of the Web 2.0 craze, but on occasion they can be helpful for taking a quick glance at the data.
Right away we see that “search” dominates along with the less helpful “better” and “easier” (imagine if you deliver a finding that said uses want things better and easier!).
Navigate also shows up, but certainly lower down in the list after things like videos and movies (which likely come from websites like Netflix and YouTube).
Content Analysis
Next we went through each response and categorized the phrases into groups to look for patterns. The graph below shows the main groups of what users said they would improve across these 30 websites.
Figure 1: Categories derived from a content analysis of open ended responses to “What one thing would you fix” across 30 websites.
For example, we grouped 21% of the responses in to a category called “More Services/Information.” Three comments and the websites they came from in this group are shown below:
What one thing would you improve on the website?
| More Services/Information | Website |
| A wider variety of products | Zappos |
| I would like more free layouts to choose from | . Vistaprint |
| More detailed descriptions of items | Target |
Again “search” shows up high in the list as we also saw in the word cloud with Yahoo, Amazon, Walmart and YouTube each generating 8% of the comments. Appearance is second with 12% of the responses mostly coming from LinkedIn, IMDB, Best Buy and Craigslist. While “appearance” didn’t make the word cloud, we do see the appearance related words of “layout, color and design.”
This again is a reminder that word clouds provide a good first pass at data but are no substitute for deeper synthesis to put words into context.
The “Security/Anti-Spam” category which comprised 8% of the responses was largely driven by websites Craigslist (40%), Facebook (29%) and PayPal (8%). Three examples in this category are shown below.
| Security/Anti Spam | Website |
| Better security controls against spammers | Yahoo |
| It is a security issue, so I would like it to be known to be secure | Craigslist |
| Higher safe guards against scammers | PayPal |
While we usually limit our analysis to within a single website, by combining 30 websites we are able to see some patterns that resonate with what we typically see in our usability studies. Problems with navigation and a poor search tend to plague the usability of websites, even highly visited ones as shown in this data.
Findability is a key component of overall website usability and we see it appearing in the list of fixes. What’s more, we also see some good evidence for why the SUPR-Q measures aspects of usability, credibility/trust and appearance. All three have prominent roles in website quality and in the list of things to fix.
So what one thing should you fix on your website?
It’s likely to be one of the items we found in our list but it will certainly depend on the website. A good first step is asking users what they’d fix. Next measure the key aspects of usability, credibility, appearance and loyalty before and after you make changes to quantify the better user experience.
