{"id":594,"date":"2020-06-10T03:45:14","date_gmt":"2020-06-10T03:45:14","guid":{"rendered":"http:\/\/measuringu.com\/leading-vs-lagging\/"},"modified":"2022-03-21T16:25:12","modified_gmt":"2022-03-21T22:25:12","slug":"leading-vs-lagging","status":"publish","type":"post","link":"https:\/\/measuringu.com\/leading-vs-lagging\/","title":{"rendered":"Leading Vs. Lagging Measures in UX"},"content":{"rendered":"<p><a href=\"https:\/\/measuringu.com\/wp-content\/uploads\/2020\/06\/Feature_061020.jpg\"><img fetchpriority=\"high\" decoding=\"async\" class=\"alignleft wp-image-19090 size-medium\" src=\"https:\/\/measuringu.com\/wp-content\/uploads\/2020\/06\/Feature_061020-300x169.jpg\" alt=\"\" width=\"300\" height=\"169\" \/><\/a>Driving down the road while only looking in the rearview mirror &#8230; that gives you a good idea of where you\u2019ve been, but unless the road behind you is exactly like the one in front of you, you may run into some obstacles, to put it mildly. Safe and effective driving means looking forward and backward.<\/p>\n<p>And the same applies to using metrics in business and in measuring the user experience. Managing by metrics is nothing new and is often used as part of a larger framework such as <a href=\"https:\/\/en.wikipedia.org\/wiki\/Total_quality_management\">Total Quality Management<\/a>, <a href=\"https:\/\/en.wikipedia.org\/wiki\/Six_Sigma\">Six Sigma<\/a>, <a href=\"https:\/\/en.wikipedia.org\/wiki\/OKR\">OKR (Objectives and Key Results)<\/a>, <a href=\"https:\/\/en.wikipedia.org\/wiki\/Management_by_objectives\">Management by Objectives<\/a>, and <a href=\"https:\/\/www.amazon.com\/Disciplines-Execution-Achieving-Wildly-Important\/dp\/145162705X\/\">Four Disciplines of Execution (4DX)<\/a>.<\/p>\n<p>This last framework is featured in the book <em>The\u00a04 Disciplines of Execution<\/em> by Sean Covey and Chris McChesney. Sean is the son of the late Steven Covey, famous for <a href=\"https:\/\/www.amazon.com\/Habits-Highly-Effective-People-Powerful\/dp\/1476740054\/\"><em>The\u00a07 Habits of Highly Effective People.<\/em><\/a><\/p>\n<p>In the 4DX book, Covey and McChesney make the distinction between metrics that are leading (looking forward) and lagging (looking backward). They emphasize the importance in management of paying attention to leading metrics that are key drivers of lagging metrics. This distinction is quite similar to the idea of <a href=\"https:\/\/measuringu.com\/understanding-variables\/\">dependent and independent variables<\/a>, a core part of most <a href=\"https:\/\/measuringu.com\/scientific-thinking\/\">scientific measurement<\/a>.<\/p>\n<p>When it comes to UX metrics it can be helpful to think of metrics not as just lagging (dependent) and leading (independent) but as a causal chain from lagging to leading, with metrics acting in intermediate steps and as both leads (inputs) and lags (outcomes). Here is a way to think about leading and lagging measures in UX measurement.<\/p>\n<h3>Ultimate Lag Measure<\/h3>\n<p>Ultimate lag measures are the outcomes you hope to achieve. They\u2019re the results of other activities and don\u2019t directly affect any subsequent measurements. These are your goals or targets, typically with a deadline set for their measurement. Once the time comes to measure them, it\u2019s too late to do anything about them other than to note whether you\u2019ve achieved your objectives and, if you haven\u2019t, to figure out what changes need to be made moving forward. Using <a href=\"https:\/\/measuringu.com\/scientific-thinking\/\">scientific thinking<\/a>, you develop hypotheses about what leading measures were likely responsible for limiting the achievement of your final lag measure goals, then develop and execute plans to improve the targeted leading measures.<\/p>\n<p>Common lag measures include:<\/p>\n<ul>\n<li>Revenue (product or business)<\/li>\n<li>Profit<\/li>\n<li>Sales<\/li>\n<li>New subscriptions (such as SaaS)<\/li>\n<li>Subscription renewals<\/li>\n<li>Recommendations and referrals<\/li>\n<li>Product usage (logins, features used)<\/li>\n<\/ul>\n<h3>Intermediate Lag Measures<\/h3>\n<p>While ultimate lag measures represent final outcomes in the causal chain, other metrics lag as outcome measures but provide a preliminary indicator of the ultimate lag measures. Some examples of intermediate lag measures include<\/p>\n<ul>\n<li><a href=\"https:\/\/measuringu.com\/nps-ux\/\">Net Promoter Score (NPS)<\/a><\/li>\n<li><a href=\"https:\/\/measuringu.com\/10-things-sus\/\">System Usability Scale (SUS)<\/a><\/li>\n<li><a href=\"https:\/\/measuringu.com\/10-things-suprq\/\">Standardized User Experience Percentile Rank Questionnaire (SUPR-Q)<\/a><\/li>\n<li><a href=\"https:\/\/measuringu.com\/umux-lite\/\">Usability Metric for User Experience Lite (UMUX-Lite)<\/a><\/li>\n<\/ul>\n<p>Metrics such as these are usually the primary output on <a href=\"https:\/\/measuringu.com\/ux-scorecard\/\">scorecards from UX benchmark studies<\/a>\u00a0and can typically be collected more frequently than ultimate lag measures. Even within this category, some intermediate lag measures may be drivers of others. For example, it\u2019s well known that <a href=\"https:\/\/measuringu.com\/nps-sus\/\">SUS and NPS are correlated<\/a>, but it\u2019s more likely that levels of perceived usability (SUS) affect subsequent likelihood-to-recommend (NPS) than for levels of likelihood-to-recommend to affect the magnitude of perceived usability. Regardless, these intermediate lag measures are clearly at a different place in the causal chain than the lower-level lead metrics that drive them. You can tell because companies can\u2019t stay in business by improving just NPS or SUS scores!<\/p>\n<h3>Secondary Lead Measures<\/h3>\n<p>Measures such as the SUS and NPS are themselves influenced by experiences. In UX research and usability engineering, these are metrics like<\/p>\n<ul>\n<li><a href=\"https:\/\/measuringu.com\/completion-rates\/\">Completion rates<\/a><\/li>\n<li><a href=\"https:\/\/measuringu.com\/errors-ux\/\">Error rates<\/a><\/li>\n<li><a href=\"https:\/\/measuringu.com\/task-times\/\">Completion times<\/a><\/li>\n<li><a href=\"https:\/\/measuringu.com\/seq10\/\">Single Ease Question (SEQ)<\/a><\/li>\n<\/ul>\n<p>These secondary lead metrics are typically the task-level metrics collected in a usability test and are themselves affected by primary lead measures.<\/p>\n<h3>Primary Lead Measures<\/h3>\n<p>Primary lead measures can be thought of as the first movers in the causal chain to ultimate UX (and often business) outcomes, and they include<\/p>\n<ul>\n<li><a href=\"https:\/\/measuringu.com\/usability-problems\/\">UI problems<\/a><\/li>\n<li><a href=\"https:\/\/measuringu.com\/prioritize-requirements\/\">Features (or lack thereof)<\/a><\/li>\n<li><a href=\"https:\/\/measuringu.com\/brand-affinity-sentiment\/\">Brand attitude<\/a><\/li>\n<li><a href=\"https:\/\/measuringu.com\/prior-exposure\/\">Prior experiences<\/a><\/li>\n<\/ul>\n<p>UI problems, software\/hardware bugs, and inadequate sets of features are the root causes for task failure, long task times, errors, and poor ratings of ease. The effects of problems, bugs, and inadequate features on downstream metrics can be moderated (made better or worse) by users\u2019 preexisting attitudes toward a company\/brand and their prior experiences with the company\/brand or its competitors.<\/p>\n<p>&nbsp;<\/p>\n<p><a href=\"https:\/\/measuringu.com\/wp-content\/uploads\/2020\/06\/LeadLag_UXMeasures.png\"><img decoding=\"async\" class=\"alignnone wp-image-19094 size-full\" src=\"https:\/\/measuringu.com\/wp-content\/uploads\/2020\/06\/LeadLag_UXMeasures.png\" alt=\"\" width=\"1939\" height=\"666\" \/><\/a><\/p>\n<p class=\"wp-caption-text\">Figure 1. Model of Primary Lead to Ultimate Lag measures.<\/p>\n<p>The figure shows that finding and fixing UI problems in a usability test (Primary Lead) will improve completion rates on tasks (Secondary Lead), which in turn improves SUS scores (Intermediate Lag), leading to more product usage (Ultimate Lag).<\/p>\n<h2>Leading and Lagging Measures in Technology Acceptance<\/h2>\n<p>The idea of using leading and lagging measures is actually quite fundamental to core principles around why people do and don\u2019t use new technology products.<\/p>\n<p>The well-known <a href=\"https:\/\/measuringu.com\/tam\/\">Technology Acceptance Model<\/a> (Figure 2) came out of information systems management research over three decades ago. It has been very influential in demonstrating the sequence of internal states of users that lead to the ultimate lag measure of Actual System Use (Attitude &gt; Intention &gt; Behavior), driven by perceptions of usefulness and ease of use. Using the nomenclature we\u2019ve adopted for this article, the causal chain from Perceived Usefulness and Perceived Ease of Use through Behavioral Intention to Use are intermediate lag measures.<\/p>\n<p><a href=\"https:\/\/measuringu.com\/wp-content\/uploads\/2020\/06\/1.png\"><img decoding=\"async\" class=\"alignleft wp-image-19069 size-full\" src=\"https:\/\/measuringu.com\/wp-content\/uploads\/2020\/06\/1.png\" alt=\"\" width=\"974\" height=\"237\" \/><\/a><\/p>\n<p class=\"wp-caption-text\">Figure 2. Technology Acceptance Model (TAM) from Davis, 1989.<\/p>\n<p>In the TAM, the leading measures are bundled together as External Variables. Much of the research using the Technology Acceptance Model has been to understand how different bundles of external variables, hypothesized to affect perceived usefulness and ease of use, actually do so. <a href=\"https:\/\/www.researchgate.net\/publication\/339333658_Perceived_Usability_and_the_Modified_Technology_Acceptance_Model\">Recent research<\/a> has shown that the metrics of the Technology Acceptance Model, when adapted to measure user experience instead of anticipated use, behave in a similar way to predict intention to recommend.<\/p>\n<h2>Applying Leading and Lagging Metrics<\/h2>\n<p>Here are three examples of using leading and lagging measures to improve user experience and ultimately impact the business.<\/p>\n<p><em>Adding a Missing Feature<\/em><\/p>\n<p>A software company we worked with added a new feature to automate the ability to integrate contacts into their sales application (primary lead). This significantly reduced the time taken to complete the adding contacts task (secondary lead). Faster task times and the inclusion of the feature improved SUS scores (intermediate lag), and the task time improvement was featured prominently in a marketing campaign to improve subscription sales (ultimate lag).<\/p>\n<p><em>Removing a Feature: Out of Sight, Out of Mind?<\/em><\/p>\n<p>In the late 1980s, a major computer manufacturer decided to cut costs by removing the hardfile in-use indicator from their floor-standing personal computers because, they reasoned, users wouldn\u2019t see it under their desks. Upon hearing this, the human factors department conducted a study with 20 randomly selected users of floor-standing computers on site, getting their permission to cover the indicator with opaque tape. After one week, seven users (35%) had removed the tape because they were annoyed by the absence of the indicator, which they had been unconsciously using as a system status check. The discovery of the issue and consequent presentation of the results were too late to stop the shipment of the product. However, having been forewarned, there was an effort to monitor users&#8217; reactions in the field, which was quickly determined to be consistent with the onsite UX study. For these reasons, the hardfile in-use indicator was returned to all following floor-standing models in that product line, and the sales group was prepared to handle customer complaints.<\/p>\n<p>In this example, the measurement of the percentage of users removing opaque tape from an indicator was a primary leading measure that was predictive of downstream customer dissatisfaction (Intermediate Lag). (For more details, see p. 120 in <a href=\"https:\/\/measuringu.com\/book\/quantifying-the-user-experiencepractical-statistics-for-user-research\/\">Quantifying the User Experience<\/a>.)<\/p>\n<p><em>Iterative Design for the Olympics<\/em><\/p>\n<p><a href=\"https:\/\/course.ccs.neu.edu\/is4300f15\/ssl\/gould.pdf\">Gould et al. (1987)<\/a> applied iterative evaluation to the design of a successful <a href=\"https:\/\/measuringu.com\/testing-effective\/\">messaging system<\/a> for the 1984 Olympics. Once released, it ran 24 hours per day for the four weeks the Olympic villages were open and was successfully used over 40,000 times during the four-week event.<\/p>\n<p>In this example, the application of an early version of <a href=\"https:\/\/measuringu.com\/ucd-and-design-thinking\/\">User-Centered Design<\/a> with mixed methods and iterative prototyping identified numerous usability issues (Primary Lead) that upon correction improved task performance (Secondary Lead), which led to a significant amount of system usage (Ultimate Lag).<\/p>\n<h2><strong>Summary and Takeaways<\/strong><\/h2>\n<p>Many UX metrics can be both lead and lag measures. For example, we\u2019ve seen how attitudes toward usability (SUS) are correlated with intent to recommend (NPS).<\/p>\n<p>The causal chain starts with primary lead measures of atomic, low-level user experiences plus prior brand attitudes and product experience, which drive secondary lead measures at the task level, which in turn drive intermediate lag measures at the product level, affecting ultimate lag measures.<\/p>\n<p>The earlier a metric is in the causal chain, the more diagnostic it tends to be. There is little diagnostic information in ultimate lag measures. That\u2019s one reason why Covey and McChesney urge managers to act on lead measures (their Discipline #2).<\/p>\n<p>It\u2019s rare for a single improvement in a primary lead measure to have an impact on ultimate lag measures. The causal chain is just too long and is often affected by non-UX-driven decisions such as price points, supply chain issues, and broad economic events (e.g., pandemics).<\/p>\n<p>Nevertheless, <a href=\"https:\/\/measuringu.com\/mixing-methods\/\">mixed-method<\/a> design and development practices such as <a href=\"https:\/\/measuringu.com\/ucd-and-design-thinking\/\">User-Centered Design, Design Thinking<\/a>, and <a href=\"https:\/\/measuringu.com\/scientific-thinking\/\">Scientific Thinking<\/a> typically bring about many improvements in primary UX lead measures, and there is <a href=\"https:\/\/measuringu.com\/testing-effective\/\">substantial evidence<\/a> that those efforts improve secondary UX lead measures, intermediate UX lag measures, and ultimate lag measures.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Driving down the road while only looking in the rearview mirror &#8230; that gives you a good idea of where you\u2019ve been, but unless the road behind you is exactly like the one in front of you, you may run into some obstacles, to put it mildly. Safe and effective driving means looking forward and [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":3448,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"site-sidebar-layout":"default","site-content-layout":"default","ast-site-content-layout":"","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"default","adv-header-id-meta":"","stick-header-meta":"default","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""}},"_price":"field_56e41332a1ae5","_stock":"","_tribe_ticket_header":"","_tribe_default_ticket_provider":"Tribe__Tickets_Plus__Commerce__WooCommerce__Main","_tribe_ticket_capacity":"0","_ticket_start_date":"","_ticket_end_date":"","_tribe_ticket_show_description":"","_tribe_ticket_show_not_going":false,"_tribe_ticket_use_global_stock":"","_tribe_ticket_global_stock_level":"","_global_stock_mode":"","_global_stock_cap":"","_tribe_rsvp_for_event":"","_tribe_ticket_going_count":"","_tribe_ticket_not_going_count":"","_tribe_tickets_list":"[]","_tribe_ticket_has_attendee_info_fields":false,"footnotes":""},"categories":[48,2,62,39],"tags":[3,56,6,207,230],"acf":[],"ticketed":false,"_links":{"self":[{"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/posts\/594"}],"collection":[{"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/comments?post=594"}],"version-history":[{"count":1,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/posts\/594\/revisions"}],"predecessor-version":[{"id":31787,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/posts\/594\/revisions\/31787"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/media\/3448"}],"wp:attachment":[{"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/media?parent=594"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/categories?post=594"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/tags?post=594"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}