Should You Care If Your Rating Scale Data Is Interval Or Ordinal?

It’s fine to compute means and statistically analyze ordinal data from rating scales. But just because one rating is twice as high as another does not mean users are really twice as satisfied. When we use rating scales in surveys, we’re translating intangible fuzzy attitudes about a topic into specific quantities. Overall, how satisfied are

Read More »

Are both positive and negative items necessary in questionnaires?

There is a long tradition of including items in questionnaires that are phrased both positively and negatively. This website was easy to use. It was difficult to find what I needed on this website. The major reason for alternating item wording is to minimize extreme response bias and acquiescent bias. However, some recent research[pdf] Jim

Read More »

Does Prior Experience Affect Perceptions Of Usability?

Are attitudes about usability constant? If we think something is unusable today, will we think it’s unusable tomorrow, next week or next year? How much does prior experience affect how usable we think a websites or software is? Enough to pay attention to. In a recent assessment, prior experience boosted usability ratings 11% for websites

Read More »

Top-Box Scoring of Rating Scale Data

Rating scales are used widely. Ways of interpreting rating scale results also vary widely. What exactly does a 4.1 on a 5 point scale mean? In the absence of any benchmark or historical data, researchers and managers look at so-called top-box and top-two-box scores (boxes refer to the response options). For example, on a five-point

Read More »

That’s The Worst Website Ever!: Effects Of Extreme Survey Items

Items in questionnaires are typically worded neutrally so as not to state concepts in the extreme. They are like an even-tempered friend—they have opinions but aren’t overly optimistic or chronically pessimistic about things. What happens when items in a questionnaire or survey are worded in the extreme? Two years ago we tried a little experiment

Read More »

Survey Respondents Prefer the Left Side of a Rating Scale

Subtle changes to response items in surveys and questionnaires can affect responses. Many of the techniques for item and scale construction in user-research come from marketing and psychology. Some topics can be controversial, sensitive or confusing and so having the right question with the right response options is important. Attitudes about usability aren’t typically controversial

Read More »

Should You Use 5 Or 7 Point Scales?

If you’ve ever designed a survey or questionnaire you’ve probably wondered how many points the response options should have. Most questionnaires I’ve examined either use five point scales or seven-point scales.  Is one better? 7-point scales are slightly better The short answer is that 7-point scales are a little better than 5-points—but not by much.

Read More »

If You Could Only Ask One Question, Use This One.

Was a task difficult or easy to complete? Performance metrics are important to collect when improving usability but perception matters just as much. Asking a user to respond to a questionnaire immediately after attempting a task provides a simple and reliable way of measuring task-performance satisfaction. Questionnaires administered at the end of a test such

Read More »

Do Users Fail A Task And Still Rate It As Easy?

Have you ever watched a user perform horribly during a usability test only to watch in amazement as they rate a task as very easy to use? I have, and as long as I’ve been conducting usability tests, I’ve heard of this contradictory behavior from other researchers. Such occurrences have led many to discount the

Read More »
0
    0
    Your Cart
    Your cart is emptyReturn to Shop
    Scroll to Top