{"id":264,"date":"2015-05-19T22:00:00","date_gmt":"2015-05-19T22:00:00","guid":{"rendered":"http:\/\/measuringu.com\/expert-review\/"},"modified":"2021-01-28T06:30:05","modified_gmt":"2021-01-28T06:30:05","slug":"expert-review","status":"publish","type":"post","link":"https:\/\/measuringu.com\/expert-review\/","title":{"rendered":"The Expert Review is More Than a Second-Rate Usability Test"},"content":{"rendered":"<p><img fetchpriority=\"high\" decoding=\"async\" style=\"float: left; padding: 10px; margin-right: 20px;\" title=\"Substitutes from http:\/\/s.hswstatic.com\/gif\/artificial-sweeteners-1.jpg\" src=\"https:\/\/measuringu.com\/images\/sweetners.jpg\" width=\"307\" height=\"173\" \/>Usability testing is an <a href=\"https:\/\/measuringu.com\/blog\/10-uxmethods.php\">essential method<\/a> to measure the user experience.<\/p>\n<p>I write a lot about the value of having actual users <a href=\"https:\/\/measuringu.com\/topics\/Usability%20Testing\">attempt tasks in an interface<\/a>&#8230;for good reason.<\/p>\n<p>Observing just a few participants attempt actual tasks can reveal a great deal about interaction problems and generate ideas on what to fix.<\/p>\n<p>But if you conduct enough usability tests and observe enough users, you notice patterns in behavior. That is, despite the many variations in websites, software products, and hardware products, we see patterns across all interfaces.<\/p>\n<p>Some examples of problems that consistently tend to cause problems for users are<\/p>\n<ul>\n<li>poorly labeled functions or menus (Where do I find X? How do I do Y?)<\/li>\n<li>lack of confirmation messages (I clicked; did it work? Is anything happening?)<\/li>\n<li>confusing error messages (What does that error code mean?)<\/li>\n<li>lack of application status (Is the app working? Is it stuck?)<\/li>\n<\/ul>\n<p>Like a physician who can diagnose an illness without having to run any tests&#8211;trained evaluators independently inspecting an interface <a href=\"https:\/\/measuringu.com\/blog\/effective-he.php\">can uncover many usability problems<\/a> without having to test any participants.<\/p>\n<p>There are a number of flavors of Expert Reviews (also called <a href=\"http:\/\/www.nngroup.com\/articles\/summary-of-usability-inspection-methods\/\">inspection methods<\/a>). Heuristic Evaluations and <a href=\"https:\/\/measuringu.com\/blog\/he-cw.php\">Cognitive Walkthroughs<\/a> are the most popular. In practice, the terms are used interchangeably and the methods are co-mingled.\u00a0 A hybrid services of the two methods we use when conducting an Expert Review that&#8217;s both efficient and effective is to:<\/p>\n<ol>\n<li>Define the users and their goals.<\/li>\n<li>Define the tasks for users to perform.<\/li>\n<li>Step through the tasks as users would: know what terms, features, and capabilities they expect, and anticipate what paths they&#8217;ll take.<\/li>\n<li>Identify, based on a set of heuristics (guidelines), known and expected issues.<\/li>\n<li>Determine the location and severity of, and list possible design fixes for, each issue.<\/li>\n<\/ol>\n<h2>Expert Review as Sub-par Substitute<\/h2>\n<p>For about as long as there have been Expert Reviews, there&#8217;s been papers asking whether Expert Reviews are a suitable replacement for usability testing:<\/p>\n<blockquote><p><a href=\"http:\/\/pro.sagepub.com\/content\/36\/4\/409.abstract\">Usability Testing vs. Heuristic Evaluation: A Head-to-Head Comparison<\/a><br \/>\n<a href=\"http:\/\/www.sciencedirect.com\/science\/article\/pii\/S016981410800053X\">Web evaluation: Heuristic evaluation vs. user testing<\/a><br \/>\n<a href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/19380955\">Comparative study of heuristic evaluation and usability testing methods<\/a><\/p><\/blockquote>\n<p>And while it&#8217;s essential to <a href=\"https:\/\/measuringu.com\/blog\/validity-research.php\">establish the validity<\/a> of a new method, the value in Expert Reviews is not that they&#8217;re an equal substitute for usability testing (they&#8217;re not), but rather they are a sufficient alternative when you can&#8217;t conduct a usability test.<\/p>\n<p>If you have the budget and time to run a usability test or Expert Review and had to pick, go with the usability test in most cases. However, where Expert Reviews become most valuable is when it&#8217;s difficult to run a usability test or where a usability test doesn&#8217;t solve all your problems.<\/p>\n<p>Here are six reasons why an Expert Review can be the right call:<\/p>\n<h2>1.\u00a0\u00a0 \u00a0It&#8217;s difficult to find users.<\/h2>\n<p>Many B2B applications\u2014and especially internal company apps\u2014often have only a few hundred users spread across timezones. Note the dilemma: the applications or website most in need of user feedback are often those with the fewest and least accessible users.<\/p>\n<p>If products are not easily accessed online, recruiting users becomes even more problematic. It can take months to get even three or four participants to take a usability test!<\/p>\n<h2>2.\u00a0\u00a0 \u00a0You don&#8217;t want to waste user time on obvious issues.<\/h2>\n<p>You find, schedule, and test those hard-to-find users and then uncover problems that were obvious. Especially during the early stages of product development (or when no prior testing has been done), an Expert Review can harvest the low-hanging fruit&#8211;save your valuable user time for later in the cycle.<\/p>\n<h2>3.\u00a0\u00a0 \u00a0Expert reviews find around 30% of the issues uncovered in a usability test.<\/h2>\n<p>We&#8217;ve found that Expert Reviews <a href=\"https:\/\/measuringu.com\/blog\/effective-he.php\">tend to find around 30% of the issues<\/a> uncovered in a usability test.\u00a0 It&#8217;s ideal to have more than one Expert Review the interface independently. And while you want experts who are familiar with the domain (e.g. financial software), we&#8217;ve found even novice evaluators uncover real problems that more seasoned experts miss.<\/p>\n<h2>4.\u00a0\u00a0 \u00a0Expert Reviews find problems that usability testing misses.<\/h2>\n<p>Just because a problem identified in an Expert Review wasn&#8217;t uncovered in a usability test doesn&#8217;t mean it&#8217;s NOT a problem. In my experience, while Expert Reviews can uncover trivial issues, few are actually not real problems (false positives).<\/p>\n<h2>5.\u00a0\u00a0 \u00a0You can&#8217;t justify the cost or time of usability testing.<\/h2>\n<p>Even if you don&#8217;t pay your participants, usability testing has a cost. The fastest usability tests require time for recruiting, time for facilitation, and time for compiling the findings. A typical Expert Review, on the other hand, takes no more than a few hours, start to finish.<\/p>\n<h2>6. A Usability test can&#8217;t cover everything.<\/h2>\n<p>Usability-test participants can attempt only so many tasks on a product in the time allotted. Thus, in many cases, the usability test reveals problems on only a small portion of the interface. Expert Reviews, at less cost, can go wide and deep, uncovering problems throughout the interface.<\/p>\n<h2>Conclusion<\/h2>\n<p>Expert Reviews aren&#8217;t a panacea.\u00a0 They aren&#8217;t a full substitute for user testing, but they&#8217;re something more than a second-rate usability test. When it&#8217;s difficult to conduct a usability test, or where a usability test won&#8217;t solve your problems, an Expert Review is often the better tool.<\/p>\n<p>A thorough review of an interface (ideally by people familiar with the domain), with multiple evaluators, focused on tasks and goals, compared to a set of guidelines, provides quick insight into potential problems and may fix more problems faster than a usability test.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Usability testing is an essential method to measure the user experience. I write a lot about the value of having actual users attempt tasks in an interface&#8230;for good reason. Observing just a few participants attempt actual tasks can reveal a great deal about interaction problems and generate ideas on what to fix. But if you [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":1728,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""}},"_price":"","_stock":"","_tribe_ticket_header":"","_tribe_default_ticket_provider":"","_tribe_ticket_capacity":"0","_ticket_start_date":"","_ticket_end_date":"","_tribe_ticket_show_description":"","_tribe_ticket_show_not_going":false,"_tribe_ticket_use_global_stock":"","_tribe_ticket_global_stock_level":"","_global_stock_mode":"","_global_stock_cap":"","_tribe_rsvp_for_event":"","_tribe_ticket_going_count":"","_tribe_ticket_not_going_count":"","_tribe_tickets_list":"[]","_tribe_ticket_has_attendee_info_fields":false,"footnotes":""},"categories":[139],"tags":[140],"acf":[],"ticketed":false,"_links":{"self":[{"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/posts\/264"}],"collection":[{"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/comments?post=264"}],"version-history":[{"count":0,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/posts\/264\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/media\/1728"}],"wp:attachment":[{"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/media?parent=264"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/categories?post=264"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/tags?post=264"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}