Figure 7:<\/strong> Participant preference for SEQ formats.<\/figcaption><\/figure>\nSummary and Takeaways<\/h2>\n
Did dropping numbers from the SEQ make a difference? Once again it turns out, not really.<\/p>\n
Across a broad suite of analyses, there were no statistically significant differences in means or top-box scores (using the standard alpha criterion of .05). Observed differences between means for the two-item formats were consistently small\u2014about 1% of the range of the scale when task data were combined, and from 1.5\u20133.6% of the range of the scale when separated by task.<\/p>\n
Hard-task top-box had the largest difference:<\/strong> The largest nominal (nonsignificant) finding in this analysis was a top-box difference between the numbered and unnumbered SEQ formats of 11.5% for the hard task when examining only the between-subjects (first task) data (Figure 6, Panel B). The difference was about half that when the analysis included all the data from the within-subjects design. Despite the lack of statistical significance for the 11.5% difference, the associated p<\/em>-value was a relatively low .10, and the 95% confidence interval ranged from \u22122.3 to 24.9%. We had a sample size of almost 300 participants for the total study, but that\u2019s cut in half for many of the between-subjects analyses, plus the top-box percentages were around 40\u201350%. Because 50% is the point of maximum binomial variability, a larger sample size might find this relatively large difference to be statistically significant.<\/p>\nNumbers may help improve response accuracy:<\/strong> Referring again to Figure 6, even though the differences were not statistically significant, when the task was easy, the percentage selection of 7 (the top box) was slightly higher when numbers were present. When the task was hard, the percentage selection of 7 was 11.5 percentage points lower when numbers were present. This suggests a working hypothesis that the presence of the number 7 led participants to be more likely to select it when the task was easy and less likely to select it when the task was hard. This behavior would lead to better discrimination between easy and hard tasks when comparing top-box scores.<\/p>\nMean differences remain small:<\/strong> Even so, it\u2019s important to keep in mind that there appear to be response behaviors that act as a counterweight to this in the lower response options such that there is little difference between the means for numbered and unnumbered scales regardless of task difficulty.<\/p>\nTaking all this into account, we can get reasonably precise estimates for means, and the mean differences were consistently small. Our estimates for top-box scores are less precise, but we find the magnitudes of the observed format differences when the task was hard to be concerning regarding the likelihood of actual equivalence for top-box scores.<\/p>\n
Most don\u2019t care or don\u2019t notice the format; those who do prefer the numbered version two to one:<\/strong> Because participants in the experiment used both formats during the task sessions, we asked them which they preferred. Among those who had a preference (only 25%), they significantly favored the version with numbers (17% to 8%).<\/p>\nTakeaway 1<\/strong>: UX practitioners can consider these SEQ formats to be \u201cmean equivalent.\u201d As long as your key dependent measure is the mean of the SEQ, you should get the same result with either format.<\/p>\nTakeaway 2<\/strong>: UX practitioners should not consider these SEQ formats to be \u201cdistribution equivalent.\u201d If your key dependent measure will be a top-box or top-two-box score based on SEQ ratings, then you can\u2019t necessarily count on getting the same result with the different formats, especially if your tasks tend to be difficult.<\/p>\nTakeaway 3<\/strong>: Unless there\u2019s a compelling reason to do otherwise, we recommend using the version with numbers. This is the more commonly used format, and it appears to have the potential for better discrimination between easy and hard tasks, and among those who had a preference, the preference for the numbered version was statistically significant and just over 2:1.<\/p>\n","protected":false},"excerpt":{"rendered":"Over the past few months, we\u2019ve conducted several studies with different versions of the seven-point Single Ease Question (SEQ\u00ae), a popular task-level metric for perceived ease-of-use. As we\u2019ve seen with other research on rating scales, response means tend to be rather stable despite often salient changes to formatting. In our earlier SEQ research, we found […]<\/p>\n","protected":false},"author":1570,"featured_media":34064,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"site-sidebar-layout":"default","site-content-layout":"default","ast-site-content-layout":"","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"default","adv-header-id-meta":"","stick-header-meta":"default","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""}},"_price":"field_56e41332a1ae5","_stock":"","_tribe_ticket_header":"","_tribe_default_ticket_provider":"Tribe__Tickets_Plus__Commerce__WooCommerce__Main","_tribe_ticket_capacity":"0","_ticket_start_date":"","_ticket_end_date":"","_tribe_ticket_show_description":"","_tribe_ticket_show_not_going":false,"_tribe_ticket_use_global_stock":"","_tribe_ticket_global_stock_level":"","_global_stock_mode":"","_global_stock_cap":"","_tribe_rsvp_for_event":"","_tribe_ticket_going_count":"","_tribe_ticket_not_going_count":"","_tribe_tickets_list":"[]","_tribe_ticket_has_attendee_info_fields":false,"footnotes":""},"categories":[62,39],"tags":[102,45,29,26],"acf":[],"ticketed":false,"_links":{"self":[{"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/posts\/33686"}],"collection":[{"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/users\/1570"}],"replies":[{"embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/comments?post=33686"}],"version-history":[{"count":27,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/posts\/33686\/revisions"}],"predecessor-version":[{"id":38285,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/posts\/33686\/revisions\/38285"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/media\/34064"}],"wp:attachment":[{"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/media?parent=33686"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/categories?post=33686"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/tags?post=33686"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}