![\"\"](\"https:\/\/measuringu.com\/wp-content\/uploads\/2019\/07\/dependent-variables-ux.jpg\")
What gets measured gets managed.<\/p>\nWhat gets measured gets managed.<\/p>\n
It\u2019s more than a truism for business executives. It\u2019s also essential for the user experience professional.<\/p>\n
In business, and UX research in particular, you don\u2019t want to bring focus to the wrong or flawed measure. It can lead to wrong decisions<\/a> and a misalignment of effort.<\/p>\n In an earlier article<\/a>, I discussed the differences between the most common variables in UX research: dependent vs. independent, latent vs. observed, and extraneous variables.<\/p>\n One of the first things to do when conducting UX research is to identify the dependent variable that best fits your needs. In this article, I\u2019ll provide some guidance for selecting the right dependent variable based on the hundreds of projects we\u2019ve conducted at MeasuringU.<\/p>\n The dependent (or outcome) variable is what we hope changes when we change something, such as fixing a design element in an interface, reordering the features on a product page, or changing a company process or policy.<\/p>\n While the term \u201cdependent variable\u201d may be unfamiliar to some, especially those without a research background, dependent variables are more easily recognized as all those metrics organizations love to track. The metrics include both broad business metrics (often called KPIs) and more specific product- and task-level metrics. All levels include a mix of attitudes (beliefs, feelings, and intentions) and behaviors.<\/p>\n <\/p>\n Business-Level Metrics<\/strong><\/p>\n I cover business metrics more extensively in Customer Analytics For Dummies<\/a>.<\/p>\n <\/p>\n Product-Level Metrics<\/strong><\/p>\n <\/p>\n Task-Level Metrics<\/strong><\/p>\n I cover both product- and task-level metrics more extensively in Benchmarking the User Experience<\/a>.<\/p>\n All these metrics makes it difficult to know where to start. Picking the right dependent variable involves a number of considerations and shares a lot in common with assessing the quality of a measure<\/a>.<\/p>\n Here are nine considerations you can take, roughly in prioritized order, to help pick the right dependent variables for your UX research.<\/p>\n Your dependent variable<\/a> should first and foremost address your research goal and hypotheses<\/a>.<\/p>\n Using customer satisfaction to see whether a button change improved signups isn\u2019t aligned to your research and won\u2019t effectively answer your research question. A better measure would be the percent of users who signed up or at least the percent of users who clicked the button since the change.<\/p>\n If you want to know whether a new product design meets users\u2019 needs, asking about usability (the SUS) is important, but it doesn\u2019t address usefulness or utility. Instead the UMUX-Lite or items from the TAM would be better questionnaires (as would other methods).<\/p>\n You can have a dependent variable that perfectly addresses your research needs but may fail to align with what your organization cares about. If your organization is in a mature market and customer retention is more important than new customer acquisition, renewal rates, attrition rates, and likelihood to repurchase are probably more important than likelihood to recommend or referral rates. Effective UX research works backwards from the organizational needs and will often dictate many of your dependent variables. Dependent variables that can both address research goals and align with business needs (and existing metrics) are the first two essential steps in picking your metrics.<\/p>\n You don\u2019t want a measure that addresses the research questions and business needs but is of poor quality. There are ways to assess the quality of the measure.<\/a>\u00a0Primarily, you should be concerned about:<\/p>\n If the purpose of a business is to create and retain a customer,<\/a>\u00a0then you want dependent variables that are ultimately meaningful to customers.<\/p>\n You can actually have a metric that perfectly aligns with an organization, addresses a research question, is psychometrically reliable<\/a> and valid<\/a>, but fails to address what\u2019s meaningful to your customer.<\/p>\n A notorious example is airlines\u2019 measuring on-time departures as a way of improving performance for customers. But pulling away from a jetway on time but sitting on the tarmac and being an hour late counted as an on-time departure. A more meaningful metric is on-time arrival (despite the challenges of weather). Of course, that too can be \u201chandled\u201d by just extending the time it takes to arrive<\/a>.<\/p>\n A more familiar issue closer to UX research is a better experience when calling customer support<\/a>. Businesses want happy customers and hope good customer support increases satisfaction, retention, and referrals.<\/p>\n Customers, of course, want their problems solved and solved quickly. Reducing time on customer support calls may be intended to measure call efficiency but not if it leads to calls being answered quickly but ineffectively. Call resolution is more meaningful than call time (although the latter is likely a good secondary measure when calls are resolved effectively).<\/p>\n You want a measure that\u2019s meaningful to your customer, but you also want your manager (and executives) to understand what your dependent variables mean. Consequently, a more widely used measure that\u2019s understood may be better (get more attention and action) than a less well understood and complex measure (even if it\u2019s \u201cbetter\u201d in some way). Often you can simplify your score when you present it using percentiles, colors, and adjectives based on historical data<\/a>. Oh, and don\u2019t just make up your thresholds of what defines \u201cgood\u201d and \u201cbad;\u201d that should be based on reference data. One of the best ways to interpret a measure is to compare<\/a>\u00a0it to published or historical data. Again another reason to work with standardized measures.<\/p>\n For example, if your organization uses the NPS but you heard on Twitter that a better measure might be an 11-item questionnaire from Gallup<\/a>, there should be solid evidence that the measure really is better to overcome its unfamiliarity. A good way to lose a seat at the table is to ignore the measures<\/a> a business uses (oh, and believe everything you read on Twitter). Your reward could be ignored research. A better services may be to use multiple measures (which we cover next).<\/p>\n In most UX research projects we conduct with clients, rarely does one metric solve all the research and business needs. We\u2019ll often have dependent variables addressing different levels. This will include business metrics such as customer satisfaction or NPS, then product-level SUS or SUPR-Q, and task-level completion rates. At the task level, we almost always include at least two dependent variables (usually completion rate and perceived ease) as it\u2019s hard to know which one will be more sensitive to design changes ahead of time. And pragmatically speaking, if you feel strongly that you have a \u201cbetter\u201d measure than one your organization is using, one of the best ways of gaining adoption is to include both measures to allow others to equate the two (and determine how one may or may not be better).<\/p>\n Dependent variables themselves can be composed of more dependent variables. For example, while companies that provide dating<\/a> apps and websites ultimately care about revenue and profits, these business-dependent variables themselves depend on customer retention and referrals. Customer retention and referrals depend on current customers trusting the quality of the people who post on these platforms. Therefore, measuring current customer trust will likely improve retention and referrals (see also the HEART<\/a> framework for integrating multiple measures) and ultimately sales and profits. Finding ways to link lower level product-dependent variables to higher level business-dependent variables (especially revenue) can be challenging, but we\u2019ve found it well worth the effort to align research to the business goals<\/a>.<\/p>\n You want a measure that addresses your research and the business’ needs, is of high quality, and is meaningful. But you also want a measure that\u2019s not a burden. A measure that is too burdensome to collect is a measure that doesn\u2019t get collected. Measures that are efficient to collect and analyze and that are \u201cgood\u201d enough are often better than measures that require special equipment, need a lot of time to administer, or are difficult to analyze.<\/p>\n For example, the lostness metric<\/a>\u00a0requires watching videos of participants (often repeatedly) and logging paths and screens. It takes time. We\u2019ve found that the simpler measure of perceived ease captures most of it.<\/p>\n Eye-tracking<\/a> where people look on a website or application generates very interesting heat maps and gaze-path videos as well as several dependent variables such as time in an area of interest and number of fixations. But eye-tracking requires expensive equipment and often quite a bit of time to analyze. But often self-reported data (did you notice this element?) or even easily collected click paths or mouse paths<\/a>\u00a0may be a sufficient dependent variable. It doesn\u2019t mean eye-tracking doesn\u2019t have its place; it just means be sure you really need the effort.<\/p>\n We\u2019ve also seen this with logging errors<\/a>\u00a0in experiences. Defining an error and recording it is both time consuming and subjective. Often just noting UI issues captures the \u201cwhat do we fix\u201d and the time, perceived ease, and completion rates provide sufficient measures that describe the task experience.<\/p>\n You should spend time thinking about the right dependent measures. But don\u2019t obsess. Is it mental effort, delight, love, loyalty, future intent, or affection? Which is better: error rate or completion rate? Should we throw out the NPS? Is lostness better than completion rate or task time?<\/p>\n There are a number of things to measure and just as many ways to measure them. But don\u2019t get metric mania by looking for the perfect metric for your project. You\u2019ll want to hone in on the right construct, but many measures correlate as they tap into similar things.<\/p>\n This holds true at the task level as well. The common metrics of completion rates, time, and errors correlate<\/a> [pdf] as an approximation for task usability. It\u2019s usually best to use multiple metrics and then average them in some reasonable way. This is what the Single Usability Metric (SUM<\/a>) does. And, of course, don\u2019t get too hung up on the \u201cright\u201d way to aggregate metrics!<\/p>\n <\/p>\n Summary and Takeaway<\/strong><\/p>\n It takes years of practice to perfect selecting the best dependent variables for UX research. But by starting with variables that address research questions, align with business needs, are psychometrically valid, and meaningful to the customer and business, you\u2019re taking an essential first step in measuring and managing the user experience.<\/p>\n <\/p>\n","protected":false},"excerpt":{"rendered":" What gets measured gets managed. It\u2019s more than a truism for business executives. It\u2019s also essential for the user experience professional. In business, and UX research in particular, you don\u2019t want to bring focus to the wrong or flawed measure. It can lead to wrong decisions and a misalignment of effort. In an earlier article, […]<\/p>\n","protected":false},"author":2,"featured_media":3107,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"site-sidebar-layout":"default","site-content-layout":"default","ast-site-content-layout":"","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"default","adv-header-id-meta":"","stick-header-meta":"default","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""}},"_price":"field_56e41332a1ae5","_stock":"","_tribe_ticket_header":"","_tribe_default_ticket_provider":"Tribe__Tickets_Plus__Commerce__WooCommerce__Main","_tribe_ticket_capacity":"0","_ticket_start_date":"","_ticket_end_date":"","_tribe_ticket_show_description":"","_tribe_ticket_show_not_going":false,"_tribe_ticket_use_global_stock":"","_tribe_ticket_global_stock_level":"","_global_stock_mode":"","_global_stock_cap":"","_tribe_rsvp_for_event":"","_tribe_ticket_going_count":"","_tribe_ticket_not_going_count":"","_tribe_tickets_list":"[]","_tribe_ticket_has_attendee_info_fields":false,"footnotes":""},"categories":[175,183,198,39],"tags":[150,40],"acf":[],"ticketed":false,"_links":{"self":[{"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/posts\/547"}],"collection":[{"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/comments?post=547"}],"version-history":[{"count":2,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/posts\/547\/revisions"}],"predecessor-version":[{"id":37293,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/posts\/547\/revisions\/37293"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/media\/3107"}],"wp:attachment":[{"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/media?parent=547"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/categories?post=547"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/tags?post=547"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n