{"id":13596,"date":"2020-12-01T19:51:40","date_gmt":"2020-12-01T19:51:40","guid":{"rendered":"https:\/\/measuringu.com\/rating-scale-best-practices-8-topics-examined-copy\/"},"modified":"2022-11-11T16:18:59","modified_gmt":"2022-11-11T23:18:59","slug":"change-verbs","status":"publish","type":"post","link":"https:\/\/measuringu.com\/change-verbs\/","title":{"rendered":"From Soared to Plummeted: Can We Quantify Change Verbs?"},"content":{"rendered":"\t\t
<\/a>Cases spike, home prices surge, and stock prices tank: we read headlines like these daily. But what is a spike<\/em> and how much is a surge<\/em>? When does something crater<\/em> versus tank<\/em> or just fall<\/em>?<\/p> Headlines are meant to grab our attention<\/a>. They often communicate the dramatic story the author wants to tell rather than what the data say.<\/p> It isn\u2019t easy to write headlines.<\/p> The Columbia Journalism Review<\/a> regularly collects ambiguous headlines, such as these from 2018:<\/p> These headlines are ambiguous due to their telegraphic style<\/a>, keeping headlines short by dropping function words<\/a> that would otherwise clarify their meaning.<\/p> Another way in which headlines can be ambiguous is in their use of verbs that describe changes in magnitude without including the actual amount of change. For example, these headlines:<\/p> As we read these headlines, verbs such as tank<\/em>, spike<\/em>, jump<\/em>, and fall<\/em> indicate a change in the magnitude of something, implying a direction for that change (increasing or decreasing) and different intensities (\u201ctank\u201d suggests a greater change than \u201cfall\u201d). It isn\u2019t clear, however, how these types of verbs are interpreted by readers. Complicating things further, those who write headlines tend to spice them up by using intense verbs to grab attention. At best this practice leads to more awareness of the news, but at worst it can instill panic and fear.<\/p> This concept is related to the ambiguity of expressions of uncertainty. Figure 1 shows a popular infographic<\/a> that graphs judgments of estimated probability for various probabilistic expressions. The figure juxtaposes the judgment of an expert CIA analyst (Sherman Kent) who made recommendations in the 1950s about the ranges that analysts should consider when using the terms, with dot plots of the judgments of 23 NATO officers. The data show the rough rank order of the terms from low to high estimated probability and the variability of the officers\u2019 judgments.<\/p> Inspired by this infographic, we decided to conduct an experiment to create a similar chart for verbs that express changes in magnitude through different directions (increasing vs. decreasing) and intensities.<\/p> In October 2020, 498 US-based online panel participants rated 12 change verbs in four contexts with one of two rating formats.<\/p> Half of the verbs are typically associated with decreasing amounts (cratered, tanked, fell, dropped, slumped, and plummeted) and the other half with increasing amounts (spiked, grew, jumped, rose, surged, and soared).<\/p> The four contexts were the COVID-19 pandemic, sports, politics, and stocks. Each respondent was randomly assigned to either a form field in which to type a number (no limit on the amount) or to a slider that ranged from 0 to 100.<\/p> Across four contexts and 12 verbs, each respondent made 48 assignments of a specific percent amount of change, one for each verb in each context. The order of presentation of contexts and verbs was randomized. Respondents also indicated how they interpreted each verb as either indicating a decrease or an increase. Figure 2 shows examples of instructions and items from the MUIQ<\/a> survey.<\/p> <\/a><\/p> <\/a><\/p> <\/a><\/p> We weren\u2019t sure how well people would understand the task despite our directions and multiple input options, so we paid particular attention to unusual response patterns, resulting in the correction of 6 cases and removal of 44 cases:<\/p> We conducted our analyses on the remaining 454 cases.<\/p> To get an overall picture of the results like those shown in Figure 1 for assigned probabilities, we graphed a dot plot of the results for each verb, averaging across contexts and item formats (Figure 3). All distributions had a right skew, and as the intensity of the verbs increased, the dispersion increased.<\/p> One of the first things we noticed was the dispersion was noticeably higher than the Sherman Kent probability estimates from Figure 1. This is likely a function of using a larger sample size, participants untrained in communicating probabilities or risk, and words that are potentially more ambiguous.<\/p> <\/a><\/p> Table 1 shows the medians, means, standard deviations, and skewness for the data in each row of Figure 3. Consistent with the right skewness of the distributions, the means were consistently higher than the medians. Dispersion in verb intensity also showed up in the standard deviations, increasing from 14.4 for \u201cfell\u201d to 21.4 for \u201csoared,\u201d and in the skewness<\/a> (deviation from symmetry), decreasing from 1.84 for \u201cfell\u201d to 0.54 for \u201csoared.\u201d<\/p> \n Figure 4 shows the 12 verbs arranged in ascending order after converting the means of verbs associated with decreasing amounts to negative numbers.<\/p><\/a><\/h1>
Experimental Design<\/h1>
Results<\/h1>
Data Cleaning<\/h2>
Distributions of Estimates<\/h2>
Means Higher than Medians<\/h2>
\n\n
\n\t \nVerb<\/th> Median<\/th> Mean<\/th> Std Dev<\/th> Skew<\/th>\n<\/tr>\n<\/thead>\n \n\t Soared<\/td> 32.5<\/td> 36.0<\/td> 21.4<\/td> 0.54<\/td>\n<\/tr>\n \n\t Tanked<\/td> 29.0<\/td> 32.3<\/td> 20.8<\/td> 0.68<\/td>\n<\/tr>\n \n\t Plummeted<\/td> 27.5<\/td> 32.6<\/td> 21.4<\/td> 0.93<\/td>\n<\/tr>\n \n\t Surged<\/td> 27.5<\/td> 31.9<\/td> 20.2<\/td> 0.73<\/td>\n<\/tr>\n \n\t Spiked<\/td> 26.8<\/td> 31.1<\/td> 20.9<\/td> 1.03<\/td>\n<\/tr>\n \n\t Cratered<\/td> 26.3<\/td> 31.5<\/td> 22.1<\/td> 0.87<\/td>\n<\/tr>\n \n\t Jumped<\/td> 16.3<\/td> 22.2<\/td> 16.9<\/td> 1.20<\/td>\n<\/tr>\n \n\t Slumped<\/td> 12.5<\/td> 16.9<\/td> 13.2<\/td> 1.25<\/td>\n<\/tr>\n \n\t Rose<\/td> 10.8<\/td> 18.6<\/td> 18.5<\/td> 1.53<\/td>\n<\/tr>\n \n\t Grew<\/td> 10.0<\/td> 18.4<\/td> 18.6<\/td> 1.58<\/td>\n<\/tr>\n \n\t Dropped<\/td> 10.0<\/td> 15.5<\/td> 14.8<\/td> 1.81<\/td>\n<\/tr>\n \n\t Fell<\/td> 10.0<\/td> 15.1<\/td> 14.4<\/td> 1.84<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/p> Verb Groupings<\/h2>