How important is the first choice users make when navigating a website?
Turns out it’s even more important than we thought.
Between 2007 and 2009, Bob Bailey and researchers examined how important the first click was in determining the ultimate success of a task.
Across a dozen studies, they found an interesting result: if a user’s first click was down one of the optimal paths in a website, 87% of them would eventually succeed.
If, however, that first click was not down the optimal path, only 46% of users would succeed. In other words, users were twice as likely to succeed if their first click was down the right path on the websites they tested.
We already know the importance of first impressions, yet this research quantified just how important it is for the optimal paths to match users’ first instincts.
We wanted to see if the same pattern held with five websites we had conducted some usability testing on over the last two years. Two of the websites were rental car websites, two were automotive informational websites and one was a large internet retailer. In total, we had 21 tasks and data from 750 users (the same users were used on both rental car websites).
For each task, we identified what we thought of as the optimal first click (in some cases there were more than one optimal paths). We then used click path data from a heatmap like the one generated from MUIQ below to determine how many users clicked down the optimal and non-optimal paths and ultimately succeeded.
The results are shown below in Table 1 below.
|Study||% Success Down Right Path||%Success Down Wrong Path||Ratio of Correct to Incorrect|
Table 1: Average success rates across tasks and studies for optimal and not optimal first choices.
On average, users were 13% more likely to succeed after clicking on the correct/optimal first click.
Somewhat surprisingly, our results didn’t match the much larger 2 to 1 ratio Bailey reported from their 12 studies. Our data showed only a slight benefit in completion rates when users made the optimal first choice. We speculated that there were a number of possible reasons.
- Determining the optimal first click is tricky: For some tasks, it was easier to determine the optimal first paths, such as down a certain navigation path. However, in many cases, users search first, and searching can be an optimal strategy depending on the quality of the terms used, the search engine algorithm, the search results page, and the link the user selects. Finally, using the clickmaps doesn’t definitively identify where the users click, and due to the variability in resolution and closely placed links, some users actions are hard to discern.
- Website & task composition: It could be that the tasks and websites we tested differed from the ones used in the Bailey research. They mostly tested government websites, whereas we used more commercially focused websites.
- Number of studies and tasks: We had less than half the number of studies as the Bailey research and subsequently fewer tasks for examining.
We wanted to investigate the relationship between first choice and ultimate success by eliminating some of these variables.
First Choices in Navigation Only
To investigate the relationship between higher success rates and the correct first choice, we looked at success rates using tree tests. Tree testing is an efficient method for testing navigation structures. It also allows us to eliminate some of the potential variables that might be masking the relationship between first choice and ultimate success.
In tree testing, search is not an option, and users can only traverse a fixed set of navigation options. Every click a user takes is appropriately recorded, making both the optimal path and actual paths users take easier to determine.
We looked at eight new studies, with data from over 1000 users and 137 tasks. The navigation structures came from websites that ranged from informational, payment, and resort to a national retailer. The same users were used on two of the resort websites and on the banking website. This information is shown in Table 2 below.
|Study #||Tree Description||Tasks||Average Success Rate||Average success from right path||Average success from wrong path||Number of primary levels||Average Ratio|
Table 2: Eight tree test studies with average task success and average success rates for users who went down the optimal first path (80%) versus going down the non-optimal path (14%).
For example, the navigation tree from the retail website had 21 tasks with an overall average success rate of 81%. The average success rate for users selecting the optimal first choice was 84%, compared to the average success of 25% for users selecting the non-optimal first choice—a ratio of 3.4 to 1.
In averaging the ratio across all studies, we found the average success rate for users who went down the correct first path was 80%, while it was only 14% for users who went down the wrong first path. In other words, users were more than six times as likely to succeed if their first choice was correct. This both confirms and amplifies the results of the Bailey research.
More Primary Levels = More Success
We also looked at the relationship between the number of first-level navigation options in each tree and the average success rate per study. Interestingly, we found a strong positive correlation (r = .82; p <.05). That is, users were more successful when the first level of the tree had more choices. While there aren’t a large number of studies examined here, these eight do suggest that navigation paths with more initial options have higher findability rates. It lends credence to the idea that flat is easier than deep in navigation design (although there are limits to this approach, not to mention space constraints). Future research would need to examine this relationship further.
There are a few conclusions we can draw from this research.
- The first choice matters a lot in navigation: If a user’s first choice is not down the right path, then the rest of the information-seeking task is an uphill battle. When we limit information-seeking tasks to just navigation structures (and exclude search and in-page navigation), users are around six times more likely to succeed if their first choice is correct.
- It can be hard to identify the optimal path: On actual website there are many paths outside of the main navigation that get users to their goals efficiently. Search and in-page navigation choices mean isolating just a navigation structure can provide only a partial picture of what actually happens in real life. This may lead to masking the actual relationship.
- Flat may be better than deep in many cases: There is some preliminary evidence that navigation structures with more primary options lead to higher success rates. More data is needed to understand the limitations of this finding.
*Note: This article was updated June, 2017 to showcase updates in technologies and methods.
|UX Measurement Boot Camp : Three Days of Intensive Training on UX Methods, Metrics and Measurement Aug. 7th-9th 2019|