It’s a lot faster to mock up an image of a webpage than to build a working webpage. Using images as prototypes allows for quick iterations and testing by having participants click on locations as if it were a real working webpage or app.
But does an image used in a click test elicit the same user behavior as a live website?
In an earlier article, we reviewed the literature on click testing and found nothing that compared clicks on images with clicks on live websites. We conducted two studies using the click-testing features of our MUiQ® platform to better understand the similarities.
In our first study of five homepages, we found generally good corroboration between images and live websites. Across hotspot regions, the average absolute difference in first-click behavior was only about 6%. Of the 20 regions, six had statistically significant differences in the percentage of first clicks, but some differences were as large as 40%. The main driver of the differences was interactive elements on websites that weren’t present in static images.
In our second study, we focused on internal product pages. Our 306 participants attempted to locate a laptop on one of three website product pages (Dell, Lenovo, HP). Participants were randomly assigned to one of the three websites and presented with either an image or a live website. Our analysis found that the first clicks were generally similar between images and live websites. The average absolute difference in first-clicking behavior was only about 8%. Of the 21 regions, six had statistically significant differences in the percentage of first clicks (no significant differences for Dell). The largest difference in first-click percentages between regions was 31%.
Both studies suggest that click testing likely provides a “good enough” option to gauge where people will click as if it were a functioning website to an approximate average precision of +/−10%.
So click testing on an image seems to provide a good proxy for where people will click (an essential effectiveness measure), but how comparable is the time it takes people to click (an essential efficiency measure) on an image versus a website?
To find out, we returned to these two studies and examined the time till the first click, which we collected with the MUiQ platform.
Study 1: Homepage Times
In the study on homepages, 130 participants were randomly assigned to attempt site-relevant tasks on the five live website homepages or images of the five homepages in a click test. There were between 55 and 62 participants in each condition. Figure 1 shows the median time till the first click with 90% confidence intervals. In four out of five websites, participants took longer on the image condition than on the live website (two-sample t-tests, all p < .03 except Disqus with p = .59).
Figure 1: Median times to first click for the homepage study with 90% confidence intervals.
Study 2: Product Page Times
In the study on product pages, participants were given one task and assigned to either the product page on the live website or the image of the page in a click-test condition. Unlike Study 1, where participants attempted multiple tasks across the websites, this study was fully between subjects. We wanted to limit participants from searching on the live website condition, so we asked them not to use search. There were between 49 and 52 participants in each condition.
Figure 2 shows the median time till the first click with 90% confidence intervals. In all three conditions, participants took longer on the image condition than on the live website. The differences were statistically significant for HP and Lenovo but not for Dell (two-sample t-tests, all p < .001 except for Dell, p = .84).
Figure 2: Median times to first click for the product page study with 90% confidence intervals.
Ratio of Times
Table 1 shows the aggregated median times (in seconds) from both studies and by condition (live site versus image). Across the eight tasks, it took on average 8.5 seconds for participants to make their first click on a live site compared to 12.5 for an image—about 50% longer (1.47 times) for images.
| Site | Live Time | Image Time | Difference | Ratio (Image/Live) |
|---|---|---|---|---|
| Creative | 10 | 15 | 5 | 1.50 |
| NASA | 4 | 8 | 4 | 2.00 |
| Disqus | 21 | 16 | –5 | 0.76 |
| IKEA | 9 | 12 | 3 | 1.33 |
| CA | 9 | 17 | 8 | 1.89 |
| Dell | 8 | 9 | 1 | 1.13 |
| HP | 3 | 11 | 8 | 3.67 |
| Lenovo | 4 | 12 | 8 | 3.00 |
| Average | 8.5 | 12.5 | 4.0 | 1.47 |
Table 1: Median first-click times between live websites and images in click tests for the same eight tasks with the ratio of live times to image times. Note that mathematically, the ratio of averages is not equal to the average of ratios when denominators are not the same.
Summary and Discussion
Across two studies comparing first click times for tasks on live and image versions of eight websites, we found:
Participants take about 50% longer to make their first click on an image of a website than the live website. In seven of eight conditions, participants took longer to click on the image compared to the live website. Six of the eight differences were statistically significant. Our best estimate is that participants take on average about 50% longer to make their first click on the image of a website.
These estimates could change with a larger sample of tasks. To date, we only have image versus live site data from eight websites, one task per site. For one site (Disqus), the time to first click was nominally faster for the image than the live site, so it’s plausible that the true percentage of sites where image clicking is faster than live sites might be greater than 1/8 (12.5%). But because a 90% adjusted-Wald confidence interval around this proportion ranges from about 1% to 43%, it’s likely that first click times will generally be faster on live sites than images.
The “why” of the difference in image/live site times to first click is an open question. At this time, we can only speculate about why participants took more time to make their first click on the images of the websites. One clear difference between images and live sites is that images present a poorer set of visual clickability cues than live sites. Some clickability cues are the same on both (e.g., links indicated with underlines or colored text), but not all. For example, it’s common for selection cursors to change state when hovering over a clickable element. As shown in Figure 3, the cursor changes from the arrow to the pointing finger when hovering over “Stories” and the text also changes to bold. On the image representation, the cursor always shows the pointing finger because, on an image, everything is clickable (and the text does not change state when the pointer is near it). This might not account for all the additional time to first click on images but seems likely to substantially contribute to it.
Figure 3: Difference in clickability cues on a live site and associated website image.
Researchers using click tests to compare images with live websites should pay more attention to click location data than to click times. The percentage difference in time to first click was greater than the percentage difference in first-click locations. Specifically, the average time difference in this analysis was about 50% compared to our previous estimate of 10% for differences in first-click location. This suggests that click-location data are more reliable than click-time data for image versus live site studies.
Bottom line: Click testing of website images can yield valuable insights regarding website usability, but researchers should be aware that differences between images and live sites can affect task completion times, especially the time to first click.
