{"id":112,"date":"2012-04-17T23:15:00","date_gmt":"2012-04-17T23:15:00","guid":{"rendered":"http:\/\/measuringu.com\/two-proportions\/"},"modified":"2022-03-21T18:27:00","modified_gmt":"2022-03-22T00:27:00","slug":"two-proportions","status":"publish","type":"post","link":"https:\/\/measuringu.com\/two-proportions\/","title":{"rendered":"6 Proportions to Compare when Improving the User Experience"},"content":{"rendered":"

<\/a>One of the simplest ways to measure any event is a binary metric coded as a 1 or 0.<\/p>\n

Such a metric represents the presence or absence of just about anything of interest: Yes\/ No,\u00a0 Pass\/ Fail, Purchase\/No Purchase, On\/Off.<\/p>\n

Fundamentally the binary system is at the heart of computing as we know it.<\/a> It also plays a critical role in user research.<\/p>\n

Binary measures can even apply to traditionally qualitative research such as noting whether a user mentions distrust of sales people or marketing materials<\/a> in an interview.<\/p>\n

By adding up all the 1’s and dividing by the total number observed you get a proportion. So if 5 out of 10 users mention distrusting marketing materials in an interview you get a proportion of 5\/10 = .5 which is often expressed as a percentage 50%.<\/p>\n

You can use proportions to help make data driven decisions just about anywhere: Which design converts more? Which product is preferred? Does the new interface have a higher completion rate? What proportion of users had a problem registering?<\/p>\n

When it comes to comparing independent proportions you’ll need to conduct a statistical test called the 2 proportion test (which is equivalent to the Chi-Square test).<\/p>\n

We talk a lot about comparing two proportions in our book Quantifying the User Experience (Chapter 5)<\/a> and recommend a slight adjustment to the typical formulas you might be familiar with so it works for small and large sample sizes.<\/p>\n

To make it easier to compare proportions, I’ve created a simple online web-calculator<\/a> which will do the statistical calculations for you. Just enter two proportions to see if the difference between them is more likely due to chance or more likely a legitimate difference.<\/p>\n

Here are 6 examples of proportions and the statistical results to get you thinking:<\/p>\n

    \n
  1. Completion Rates<\/span>: If 11 out 12 users complete a task on Design A and only 5 out of 10 can complete the same task on Design B, then we can be 97% confident more users can complete the task on Design A.<\/a><\/li>\n
  2. Conversion Rates<\/span>:\u00a0 A large blue button was shown to 455 users and 37 (8%) purchased a product. A large red button was shown to 438 different users and 22 purchased the product (5%). There is a 94% chance the blue button will sell more products.<\/li>\n
  3. Problem Occurrence<\/span>: 4 out of 7 users received at least one error message when entering alerts and notifications into their profile on a credit card website. After a redesign, 1 out of 7 had at least one error. There is an 89% chance the number of errors has been reduced when setting account alerts.<\/li>\n
  4. Proportion Recommending<\/span>: 89 out of 100 (93%) customers said they recommended Smart Phone A to a friend in the last year compared to 67 out of 93 (72%) for Smart Phone B.\u00a0 There is a 99.7% chance this retroactive recommend rate is different.<\/li>\n
  5. Proportion detracting<\/span>: Prior to the change in the return policy, 49 out of 100 (49%) customers surveyed were detractors<\/a>. After the change in policy 40 out of 96 (42%) were.\u00a0 There is about a 69% chance the difference is not due to chance (good, but not overwhelming evidence).<\/li>\n
  6. Proportion that completed a task in less than 30 seconds<\/span>: 4 out of 9 users could add a new contact in CRM application A in less than 30 seconds. Eleven out of 12 could on CRM application B. There is a 97% chance if we tested all users, more would complete the tasks on App B.<\/li>\n<\/ol>\n","protected":false},"excerpt":{"rendered":"

    One of the simplest ways to measure any event is a binary metric coded as a 1 or 0. Such a metric represents the presence or absence of just about anything of interest: Yes\/ No,\u00a0 Pass\/ Fail, Purchase\/No Purchase, On\/Off. Fundamentally the binary system is at the heart of computing as we know it. It […]<\/p>\n","protected":false},"author":2,"featured_media":11227,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"site-sidebar-layout":"default","site-content-layout":"default","ast-site-content-layout":"","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"default","adv-header-id-meta":"","stick-header-meta":"default","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""}},"_price":"field_56e41332a1ae5","_stock":"","_tribe_ticket_header":"","_tribe_default_ticket_provider":"Tribe__Tickets_Plus__Commerce__WooCommerce__Main","_tribe_ticket_capacity":"0","_ticket_start_date":"","_ticket_end_date":"","_tribe_ticket_show_description":"","_tribe_ticket_show_not_going":false,"_tribe_ticket_use_global_stock":"","_tribe_ticket_global_stock_level":"","_global_stock_mode":"","_global_stock_cap":"","_tribe_rsvp_for_event":"","_tribe_ticket_going_count":"","_tribe_ticket_not_going_count":"","_tribe_tickets_list":"[]","_tribe_ticket_has_attendee_info_fields":false,"footnotes":""},"categories":[82,21],"tags":[83,22],"acf":[],"ticketed":false,"_links":{"self":[{"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/posts\/112"}],"collection":[{"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/comments?post=112"}],"version-history":[{"count":1,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/posts\/112\/revisions"}],"predecessor-version":[{"id":32004,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/posts\/112\/revisions\/32004"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/media\/11227"}],"wp:attachment":[{"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/media?parent=112"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/categories?post=112"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/tags?post=112"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}