{"id":576,"date":"2020-02-05T04:45:50","date_gmt":"2020-02-05T04:45:50","guid":{"rendered":"http:\/\/measuringu.com\/scales-cultural-effects\/"},"modified":"2023-02-13T13:25:52","modified_gmt":"2023-02-13T20:25:52","slug":"scales-cultural-effects","status":"publish","type":"post","link":"https:\/\/measuringu.com\/scales-cultural-effects\/","title":{"rendered":"Cultural Effects on Rating Scales"},"content":{"rendered":"
Numbers are universally understood across cultures, geography, and languages.<\/p>\n
But when those numbers are applied to sentiments (for example, satisfaction<\/a>, agreement, or intention), do people respond universally or does a 4 on a five-point scale elicit different reactions based on culture or geography?<\/p>\n Many international organizations use similar sets of measures (such as satisfaction or the Net Promoter Score) to compare countries and regions. If cultural differences do have a strong impact on scores it can be difficult to disentangle the effects of meaningful differences across regions (e.g., lower product satisfaction) from cultural differences, which is a form of measurement error.<\/p>\n Many published papers have identified effects from culture and geography. For example, Zax and Takahashi (1967<\/a>) found cultural and gender differences between U.S. and Japanese respondents in response patterns of seven-point semantic differential ratings of characteristics of Rorschach inkblots.<\/p>\n They found U.S. respondents were 41% more likely to select the extreme responses compared to Japanese respondents<\/strong> (19.2% vs. 13.6% respectively). Conversely, Japanese respondents selected the neutral response 33% more (23.2% vs. 17.4%).<\/p>\n The preference for extreme responses has also been seen in other western countries. For example, Dolnicar and Gr\u00fcn (2007)<\/a> found that Australian respondents selected extreme responses more than Asian respondents (Chinese, Indonesian, Indian, and Malaysian) across multiple studies.<\/p>\n Hui and Triandis (1989<\/a>) reported that Hispanic-American navy recruits who rated characteristics of supervisors were more than twice as likely as non-Hispanic recruits to select an extreme response when a five-point scale was used (but not on a ten-point scale).<\/p>\n Re-analyzing data from European surveys on cooking, cleaning, and shaving (five-point scales), Van Herk, Poortinga, and Verhallen (2004)<\/a> found that people in southern Europe (Italy, Spain, and Greece) had stronger acquiescence and extreme-response tendencies than northern European respondents (UK, Germany, and France). These findings were somewhat surprising because extreme responding is usually associated with individualistic cultures (UK, Germany, France) rather than more collectivist cultures (Italy, Spain, Greece), but Harzing (2006)<\/a> replicated the results.<\/p>\n Niikura (1999)<\/a> studied responses of white-collar workers to a questionnaire on modes of expression in the workplace, finding that U.S. respondents reported a more assertive style than Japanese, Malaysian, and Filipino respondents. This research did not, however, provide any analysis of extreme or neutral response style to the questionnaire\u2019s items.<\/p>\n Yang et al. (2010<\/a>) summarized the response-style effects by culture but concluded that the exact impact of culture on responses may be hard to predict, as it\u2019s affected by variables such as the question content and respondent motivation.<\/p>\n Yu & Yang (2015<\/a>) conducted an international comparison using Gallup\u2019s Customer Engagement Metrics CE11<\/a> across dozens of countries for both B2B and B2C brands. The CE11\u2019s eleven five-point items provide a measure of brand affinity and include items such as satisfaction and likelihood to recommend. In their study (see their Table 2), they showed how the CE11\u2019s global mean score differed by country despite ostensibly measuring the same things. The range was almost a full point (high of 4.35 in the U.S. for B2C down to 3.43 in Hong Kong for Cantonese B2C). Other notable lower-scoring countries included 3.59 B2C in Japan, 3.81 in B2B France, and 3.88 in B2C Germany. They established patterns of mean differences in responses for different cultures but did not provide data about tendencies toward neutral or extreme responding.<\/p>\n We decided to conduct our own study to see whether we could replicate<\/a> the effects found in Zak and Takahashi and Yang et al. using two commonly used scales in UX research: the eleven-point likelihood to recommend<\/a> (LTR) item and the seven-point Single Ease Question<\/a> (SEQ). We’ll examine both means and neutral\/extreme responses.<\/p>\n In September 2019, we asked 61 participants from the U.S., 51 from Japan, 53 from Germany, and 50 from France to respond to three item types (six items total):<\/p>\n We selected mobile phone carriers and restaurants as products and experiences that were common in all countries tested. For the ease question, we wanted to use a task comparable across cultures that could be universally understood but also calibrated for difficulty. We selected two math items from the Wonderlic<\/a> pre-hiring assessment (Figure 4).<\/p>\n The questions were drafted in English for the United States and translated for participants in other countries (Japanese, German, and French). We selected these countries based on the findings in Yang (2015), which showed that these countries had large differences in CE11 scores compared to U.S. respondents.<\/p>\n <\/p>\nCultural Scale Differences Study<\/h2>\n
\n