{"id":602,"date":"2020-08-12T02:13:58","date_gmt":"2020-08-12T02:13:58","guid":{"rendered":"http:\/\/measuringu.com\/entertainment-benchmarks-2020\/"},"modified":"2022-03-21T16:27:12","modified_gmt":"2022-03-21T22:27:12","slug":"entertainment-benchmarks-2020","status":"publish","type":"post","link":"https:\/\/measuringu.com\/entertainment-benchmarks-2020\/","title":{"rendered":"The UX of Video Streaming Entertainment Websites &#038; Apps"},"content":{"rendered":"<p><a href=\"https:\/\/measuringu.com\/wp-content\/uploads\/2020\/08\/Feature081320.jpg\"><img fetchpriority=\"high\" decoding=\"async\" class=\"alignleft wp-image-19652 size-medium\" src=\"https:\/\/measuringu.com\/wp-content\/uploads\/2020\/08\/Feature081320-300x169.jpg\" alt=\"\" width=\"300\" height=\"169\" \/><\/a>We\u2019ve all spent a lot of time at home this year.<\/p>\n<p>The pandemic has made <a href=\"https:\/\/www.statista.com\/outlook\/206\/100\/video-streaming--svod-\/worldwide\">already-popular<\/a> video streaming services seem essential.<\/p>\n<p>The popularity makes sense given the relatively inexpensive subscription fees, the lack of long-term contracts, and the many channels of access (through websites, mobile apps, smart TVs), and there is a LOT of content (albeit distributed across different services).<\/p>\n<p>But no matter how good the content may be, if there is friction in accessing content, then subscribers may decide to switch to a service with a better overall user experience.<\/p>\n<p>To better understand these services, we benchmarked the user experience of the following five U.S.-based streaming entertainment websites and mobile apps to see how people are using these services and where they can be improved:<\/p>\n<ul>\n<li>Disney Plus (disneyplus.com)<\/li>\n<li>HBO Now (hbonow.com)<\/li>\n<li>Hulu (hulu.com)<\/li>\n<li>Netflix (netflix.com)<\/li>\n<li>Prime Video (primevideo.com)<\/li>\n<\/ul>\n<p>We collected <a href=\"https:\/\/measuringu.com\/product\/suprq\/\">SUPR-Q data<\/a>, including <a href=\"https:\/\/measuringu.com\/nps-reliability\/\">Net Promoter<\/a>\u00a0scores, investigated reasons for using the services, measured users\u2019 attitudes toward the services, and analyzed problems users reported having with the websites\/apps.<\/p>\n<h2>Benchmark Study Details<\/h2>\n<p>In <strong>June\u2013July 2020, we had 328 respondents participate in a <\/strong><a href=\"https:\/\/measuringu.com\/benchmark-intro\/\"><strong>retrospective study<\/strong><\/a> in which we asked people who used at least one streaming entertainment service to reflect on their most recent experiences.<\/p>\n<p>Participants completed the eight-item <a href=\"https:\/\/measuringu.com\/10-things-suprq\/\">SUPR-Q<\/a> (including the <a href=\"https:\/\/measuringu.com\/nps-reliability\/\">Net Promoter Score<\/a>) and the\u00a0<a href=\"https:\/\/www.researchgate.net\/publication\/339333658_Perceived_Usability_and_the_Modified_Technology_Acceptance_Model\">UMUX-Lite<\/a> standardized questionnaires and answered questions about brand attitudes, usage, and their prior experiences. Participants who reported accessing the services on their mobile devices also answered the SUPR-Qm.<\/p>\n<h2>Quality of the Video Streaming Entertainment Website\/App User Experience: SUPR-Q<\/h2>\n<p>The SUPR-Q is a standardized measure of the quality of a website\u2019s user experience and is a good way of gauging users\u2019 attitudes. It\u2019s based on a rolling database of around 200 websites across dozens of industries, including entertainment services.<\/p>\n<p>Scores are percentile ranks and tell you how a website experience ranks relative to the other websites. The SUPR-Q provides an overall score as well as detailed scores for subdimensions of trust, usability, appearance, and loyalty.<\/p>\n<p>The scores for the five entertainment websites\/apps were mostly above average. The average is at the 72<sup>nd<\/sup> percentile (scoring better than 72% of the websites in the database). When we ran a <a href=\"https:\/\/measuringu.com\/entertainment-benchmarks\/\">similar study in 2017<\/a>, HBO GO had the lowest SUPR-Q, and Netflix had one of the highest. In this study, HBO Now has the lowest SUPR-Q of the group with a score at the 43<sup>rd<\/sup> percentile. Netflix and Disney Plus lead the group with scores at the 99<sup>th<\/sup> and 93<sup>rd<\/sup> percentiles.<\/p>\n<h3>Usability Scores and Trust<\/h3>\n<p>We asked participants to reflect on how easy they thought it was to use and navigate the entertainment websites\/apps. Netflix has the highest score in the group (99<sup>th<\/sup> percentile) and HBO Now has the lowest (20<sup>th<\/sup> percentile).<\/p>\n<p>The pattern of trust scores is a little different, with Disney Plus and Netflix scoring high at the 90<sup>th<\/sup> and 94<sup>th<\/sup> percentiles, respectively, and Prime Video scoring the lowest at the 61<sup>st<\/sup> percentile.<\/p>\n<h3>Loyalty\/Net Promoter Scores<\/h3>\n<p>The entertainment websites\/apps have an average NPS of about 9%, so there are generally more promoters than detractors across the services. In general, better user experience is a <a href=\"https:\/\/measuringu.com\/usability-loyalty\/\">good predictor of higher loyalty<\/a>.<\/p>\n<p>The correlation between overall SUPR-Q and NPS for the five services was reasonably high (r(3) = .815, p &lt; .10). The correlation between brand attitude and NPS was highly significant (r(3) = .95, p = .01).<\/p>\n<h3>SUPR-Qm Ratings of Mobile Apps<\/h3>\n<p>Respondents who reported using mobile apps to access their streaming video service completed the <a href=\"http:\/\/uxpajournal.org\/supr-qm-measure-mobile-ux\/\">Standardized User Experience Percentile Rank Questionnaire for Mobile (SUPR-Qm)<\/a>. The SUPR-Qm is a 16-item standardized questionnaire that provides a single measure of the quality of user experience with a mobile app. The 16 items form an ordered scale, from items that are easy to agree with (\u201cThe app is easy to use\u201d) to those that only someone who had an exceptional user experience would agree with (\u201cI can\u2019t live without the app on my phone\u201d).<\/p>\n<p>Overall mean SUPR-Qm ratings can range from 1 to 5. Across these entertainment services, the overall mean was 3.57 (90% confidence interval ranging from 3.47\u20133.67), statistically above the typical average from our database of dozens of mobile apps (p &lt; .10). There were significant differences among the SUPR-Qm means of the entertainment services, with Netflix having the highest mean (3.95) and HBO Now and Prime Video having the lowest means (3.30 and 3.40 respectively).<\/p>\n<h3>Use of Entertainment Sites and Mobile Apps<\/h3>\n<p>As a part of this benchmark, we asked participants how they accessed their subscriptions and their top tasks. Most participants reported using their desktop\/laptop; fewer than half reported using a mobile website. About two-thirds reported accessing content with a smart TV or mobile app.<\/p>\n<p>The typical frequency of use was a few times a week. The top tasks were watching content or browsing for content (52% and 10%, respectively).<\/p>\n<h2>Components of the Video Streaming Entertainment Experience<\/h2>\n<p>Across the sites, 81% of participants agreed there was a good selection of kids and family content. Overall, only 58% of users think it\u2019s easy to find new movies and shows. Other potential areas of improvement include loading times and settings management (especially parental controls).<\/p>\n<p style=\"padding-left: 25px;\"><em>\u201cSome problems with loading and the fact that I can\u2019t skip intros.\u201d\u2014<\/em>HBO Now user<\/p>\n<p style=\"padding-left: 25px;\"><em>\u201cDifficult to locate show; nothing arranged logically.\u201d\u2014Hulu user<\/em><\/p>\n<p style=\"padding-left: 25px;\"><em>\u201cIt\u2019s a little cluttered\u2014not terrible but not very clean either.\u201d\u2014Prime Video user<\/em><\/p>\n<h2>Key Drivers<\/h2>\n<p>To better understand what affects SUPR-Q scores, we asked respondents to agree\/disagree with the following attributes of their entertainment website on a five-point scale (from 1 = strongly disagree to 5 = strongly agree.) A <a href=\"https:\/\/measuringu.com\/key-drivers\/\">key driver analysis<\/a> uses regression modeling to quantify the extent to which ratings on these items drive (account for) variation in overall SUPR-Q scores.<\/p>\n<ol>\n<li>The website has good recommendations when I\u2019m looking for something to watch.<\/li>\n<li>It\u2019s easy to find new shows and movies.<\/li>\n<li>It\u2019s easy to navigate.<\/li>\n<li>It\u2019s easy to watch on all devices.<\/li>\n<li>The website is always fast and responsive.<\/li>\n<li>I don\u2019t experience long loading times when using the website.<\/li>\n<li>It\u2019s easy to manage settings on the website.<\/li>\n<li>It\u2019s easy to manage family profiles on the website.<\/li>\n<li>It\u2019s offered at a good price.<\/li>\n<li>The website has a good selection of original content.<\/li>\n<li>The website has shows and movies you can\u2019t find anywhere else.<\/li>\n<li>It\u2019s easy to manage parental controls.<\/li>\n<li>There\u2019s a lot of kids and family content to choose from.<\/li>\n<li>I don\u2019t have to watch commercials or ads.<\/li>\n<li>I like it better than cable or satellite TV.<\/li>\n<\/ol>\n<h3>Key Drivers Include Brand Attitude, Watching on All Devices, and Load Times<\/h3>\n<p>We often find brand attitude to be a major driver of SUPR-Q scores, and streaming services are no exception, with brand attitude accounting for 12% of the variation in overall SUPR-Q. While notable, this is of limited value because brand attitude itself is more of a <a href=\"https:\/\/measuringu.com\/leading-vs-lagging\/\">lagging than a leading indicator<\/a>. Knowing that brand attitude affects SUPR-Q ratings doesn\u2019t provide much specific guidance about what improvements to make.<\/p>\n<p>The top key driver from the list above was \u201cIt\u2019s easy to watch on all devices\u201d (accounting for 14% of SUPR-Q variation). Overall ratings of this item were moderately high, with Disney Plus and Netflix receiving the highest ratings.<\/p>\n<p>The second biggest driver, accounting for 11% of SUPR-Q variation, was \u201cI don\u2019t experience long loading times when using the website.\u201d As noted above, this component of the streaming entertainment experience had relatively low ratings across the services.<\/p>\n<p>Other items that accounted for a significant amount of variation in the SUPR-Q were \u201cThere\u2019s a lot of kids and family content to choose from\u201d and \u201cI don\u2019t have to watch commercials or ads.\u201d<\/p>\n<h3>What to Improve<\/h3>\n<p>The main areas of improvement for each website\/app were<\/p>\n<ul>\n<li><strong>Disney Plus<\/strong>: Poor selection of content (e.g., <em>\u201cI dislike how the number of shows or movies is slim compared to Netflix, but I imagine the gap to close as years pass.\u201d<\/em>)<\/li>\n<li><strong>HBO Now<\/strong>: Dislike the design (e.g., <em>\u201cI think that the interface was not refined, in comparison with Netflix.\u201d<\/em>)<\/li>\n<li><strong>Hulu<\/strong>: Too many ads (e.g., <em>\u201cHow many ads there are even with a paid service.\u201d<\/em>)<\/li>\n<li><strong>Netflix<\/strong>: Autoplay videos (e.g., <em>\u201cI dislike how the website makes videos automatically play, as it is distracting and can make you watch something you don\u2019t want to watch.\u201d<\/em>)<\/li>\n<li><strong>Prime Video<\/strong>: Unclear what content is included with Prime for free (e.g., <em>\u201cHard to tell the difference quickly if a movie is free or not, hard for older people.\u201d<\/em>)<\/li>\n<\/ul>\n<h2>Summary<\/h2>\n<p>An analysis of the user experience of five video streaming entertainment websites\/apps found that<\/p>\n<ol>\n<li><strong> Most of these entertainment websites\/apps had above average UX. <\/strong>Except for HBO Now (43<sup>rd<\/sup> percentile), the services scored at or above the 55<sup>th<\/sup> percentile in the SUPR-Q (99<sup>th<\/sup> for Netflix). NPS reflects this pattern, and only HBO Now had a negative score.<\/li>\n<li><strong> Users appreciated having a good selection of kids and family content but reported issues finding new movies and shows.<\/strong> The reported difficulties for finding new movies and shows are especially problematic given that the top tasks were watching content or browsing for content.<\/li>\n<li><strong> Ease of watching on all devices and loading times are key drivers of UX.<\/strong> In addition to brand attitude, other key drivers were the availability of family content, the presence of ads, and ease of use.<\/li>\n<\/ol>\n<p>Full details are available in the <a href=\"https:\/\/measuringu.com\/product\/ux-benchmark-entertainment\/\">downloadable report<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>We\u2019ve all spent a lot of time at home this year. The pandemic has made already-popular video streaming services seem essential. The popularity makes sense given the relatively inexpensive subscription fees, the lack of long-term contracts, and the many channels of access (through websites, mobile apps, smart TVs), and there is a LOT of content [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":3496,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"site-sidebar-layout":"default","site-content-layout":"default","ast-site-content-layout":"","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"default","adv-header-id-meta":"","stick-header-meta":"default","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""}},"_price":"field_56e41332a1ae5","_stock":"","_tribe_ticket_header":"","_tribe_default_ticket_provider":"Tribe__Tickets_Plus__Commerce__WooCommerce__Main","_tribe_ticket_capacity":"0","_ticket_start_date":"","_ticket_end_date":"","_tribe_ticket_show_description":"","_tribe_ticket_show_not_going":false,"_tribe_ticket_use_global_stock":"","_tribe_ticket_global_stock_level":"","_global_stock_mode":"","_global_stock_cap":"","_tribe_rsvp_for_event":"","_tribe_ticket_going_count":"","_tribe_ticket_not_going_count":"","_tribe_tickets_list":"[]","_tribe_ticket_has_attendee_info_fields":false,"footnotes":""},"categories":[57,226,198,39],"tags":[162,76,56,193,63],"acf":[],"ticketed":false,"_links":{"self":[{"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/posts\/602"}],"collection":[{"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/comments?post=602"}],"version-history":[{"count":1,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/posts\/602\/revisions"}],"predecessor-version":[{"id":31794,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/posts\/602\/revisions\/31794"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/media\/3496"}],"wp:attachment":[{"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/media?parent=602"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/categories?post=602"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/measuringu.com\/wp-json\/wp\/v2\/tags?post=602"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}