It seems like there are endless ways to ask questions of participants in surveys. Variety in question types can be both a blessing and a curse. Having many ways to ask questions provides better options to the researcher to assess the opinion of the respondent. But the wrong type of question can fail to capture
What does 4.1 on a 5-point scale mean? Or 5.6 on a 7-point scale? Interpreting rating scale data can be difficult in the absence of an external benchmark or historical norms. A popular technique used often by marketers to interpret rating scale data is the so-called “top box” and “top-two box” scoring approach. For example, on
Know your data. When measuring the customer experience, one of the first things you need to understand is how to identify and categorize the data you come across. It’s one of the first things covered in our UX Boot Camp and it’s something I cover in Chapter 2 of Customer Analytics for Dummies. Early consideration
The Single Ease Question (SEQ) is a 7-point rating scale to assess how difficult users find a task. It’s administered immediately after a user attempts a task in a usability test. After users attempt a task, ask them this simple question: Overall, how difficult or easy was the task to complete? Use the seven point
What makes a successful website? There are some obvious metrics like revenue, traffic and repeat visitors. But these are outcome measures. They don’t tell you why revenue or traffic is higher or lower. Key drivers of these outcomes are how the users perceive and interact with your website. Selling a product that has demand or
It’s fine to compute means and statistically analyze ordinal data from rating scales. But just because one rating is twice as high as another does not mean users are really twice as satisfied. When we use rating scales in surveys, we’re translating intangible fuzzy attitudes about a topic into specific quantities. Overall, how satisfied are
Few things tend to generate more heated debate than the format of response options used in surveys. Right in the middle of that debate is whether the number of options should be odd or even. Odd numbered response scales include a neutral response whereas even ones do not. Research generally shows that including a neutral
Rating scales are used widely. Ways of interpreting rating scale results also vary widely. What exactly does a 4.1 on a 5 point scale mean? In the absence of any benchmark or historical data, researchers and managers look at so-called top-box and top-two-box scores (boxes refer to the response options). For example, on a five-point
Items in questionnaires are typically worded neutrally so as not to state concepts in the extreme. They are like an even-tempered friend—they have opinions but aren’t overly optimistic or chronically pessimistic about things. What happens when items in a questionnaire or survey are worded in the extreme? Two years ago we tried a little experiment
Subtle changes to response items in surveys and questionnaires can affect responses. Many of the techniques for item and scale construction in user-research come from marketing and psychology. Some topics can be controversial, sensitive or confusing and so having the right question with the right response options is important. Attitudes about usability aren’t typically controversial
