{"id":38703,"date":"2023-08-08T18:49:12","date_gmt":"2023-08-09T00:49:12","guid":{"rendered":"https:\/\/measuringu.com\/?p=38703"},"modified":"2023-08-08T18:49:12","modified_gmt":"2023-08-09T00:49:12","slug":"removing-the-neutral-response-option","status":"publish","type":"post","link":"https:\/\/measuringu.com\/removing-the-neutral-response-option\/","title":{"rendered":"Does Removing the Neutral Response Option Affect Rating Behavior?"},"content":{"rendered":"

\"\"<\/a>Many topics about the design of rating scales can provoke strong opinions and heated debates<\/a>. The arguments include whether or not scales should include a neutral response.<\/p>\n

Unlike rating scales with an even number of points (e.g., 4, 6, or 10), rating scales with an odd number of points (e.g., 5, 7, or 11) contain a middle, presumably neutral, point.<\/p>\n

Researchers who want to avoid a neutral response option are concerned that having a neutral point attracts respondents who slightly lean toward a favorable or unfavorable response but don\u2019t want to commit. Having a neutral response would then mask these sentiments. But with an even number of options, respondents are forced to indicate a favorable or negative attitude when their actual sentiment is neutral.<\/p>\n

Most standardized UX questionnaires (e.g., SUS<\/a>, PSSUQ<\/a>, SUPR-Q<\/a>\u00ae<\/sup>, SEQ<\/a>\u00ae<\/sup>, UX-Lite<\/a>\u00ae<\/sup>) have items with odd numbers of points. It seems reasonable that people could genuinely have a neutral attitude toward items that ask about usability or general satisfaction with a system\u2014there’s less incentive to hide sensitive beliefs in the neutral zone than when responding to social or political questions.<\/p>\n

In a previous study<\/a>, we found a small (1.4%) difference in means when the neutral point was removed from the eleven-point Likelihood-to-Recommend item used to compute the Net Promoter Score. In that within-subjects experimental design, a ten-point version (no neutral point) had a slightly lower mean versus the standard eleven-point version (with a neutral point) and a very slightly higher top-box score (0.2%). Neither result was statistically significant.<\/p>\n

We were curious what would happen if we did this kind of manipulation to four items that we often use in our research (the SEQ, one item; the UX-Lite, two items; and the LTR, one item) if we also systematically manipulated task difficulty, so we ran an experiment to find out.<\/p>\n

Experimental Design<\/h1>\n

Using our MUiQ\u00ae<\/sup> platform<\/a> for conducting unmoderated remote UX studies, we set up an experiment to compare ratings of participant experiences in which the number of scale points included or did not include a middle (neutral) response option\u2014in other words, we changed whether the number of scale points was odd (the standard format for these items) or even (an alternate format).<\/p>\n

The Greco-Latin design.<\/em> We set up a Greco-Latin<\/a> experimental design to support a within-subjects comparison of two item formats when rating experiences with easy and hard tasks. We collected data from 200 participants (sampled in February 2023 from a U.S. panel provider).<\/p>\n

In this experimental design, there were three independent variables:<\/p>\n