MeasuringU

What Do You Gain from Larger-Sample Usability Tests?

By Jeff Sauro, PhD and Jim Lewis, PhD . September 1, 2020 . Tags: Sample Sizes, Formative, Summative, SEQ, Rating Scale, Sample Size

We typically recommend small sample sizes (5–10) for conducting iterative usability testing meant to find and fix problems (formative evaluations). For benchmark or comparative studies, where the focus is on detecting differences or estimating population parameters (summative evaluations), we recommend using larger sample sizes (20–100+). Usability testing can be used to uncover problems and assess the experience. Many usability tests will play both roles simultaneously, formative

Sample Size Recommendations for Benchmark Studies

By Jeff Sauro, PhD and Jim Lewis, PhD . June 23, 2020 . Tags: Sample Sizes, SEQ, Sample Size, SUS

One of the primary goals of measuring the user experience is to see whether design efforts actually make a quantifiable difference over time. A regular benchmark study is a great way to institutionalize the idea of quantifiable differences. Benchmarks are most effective when done at regular intervals (e.g., quarterly or yearly) or after significant design or feature changes. A UX benchmark is something akin to

Approximating Task Completion When You Can’t Observe Users

By Jeff Sauro, PhD and Jim Lewis, PhD . April 14, 2020 . Tags: Unmoderated Research, SEQ, SUS

If users can’t complete a task, not much else matters. Consequently, task completion is one of the fundamental UX measures and one of the most commonly collected metrics, even in small-sample formative studies and studies of low-fidelity prototypes. Task completion is usually easy to collect, and it’s easy to understand and communicate. It’s typically coded as a binary measure (success or fail) dependent on a participant

Cultural Effects on Rating Scales

By Jeff Sauro, PhD . February 4, 2020 . Tags: LTR, Satisfaction, SEQ

Numbers are universally understood across cultures, geography, and languages. But when those numbers are applied to sentiments (for example, satisfaction, agreement, or intention), do people respond universally or does a 4 on a five-point scale elicit different reactions based on culture or geography? Many international organizations use similar sets of measures (such as satisfaction or the Net Promoter Score) to compare countries and regions. If

Using Task Ease (SEQ) to Predict Completion Rates and Times

By Jeff Sauro, PhD . October 30, 2018 . Tags: Validity, SEQ, Questionnaire

Our attitudes both reflect and affect our actions. What we think affects what we do and what we do affects what we think. It’s not a perfect relationship of course. What people say or think doesn’t always directly correspond to actions in easily predictable ways. Understanding and measuring user attitudes early and often can provide a good indication of likely future behaviors. Attitudes can be

10 Things To Know About The Single Ease Question (SEQ)

By Jeff Sauro, PhD . October 30, 2012 . Tags: SEQ, Rating Scale

The Single Ease Question (SEQ) is a 7-point rating scale to assess how difficult users find a task. It's administered immediately after a user attempts a task in a usability test. After users attempt a task, ask them this simple question: Overall, how difficult or easy was the task to complete? Use the seven point rating scale format below. Labels and values: We typically label

Do Users Fail A Task And Still Rate It As Easy?

By Jeff Sauro, PhD . October 9, 2009 . Tags: SEQ, Usability Metrics, Rating Scale, Questionnaire

Have you ever watched a user perform horribly during a usability test only to watch in amazement as they rate a task as very easy to use? I have, and as long as I've been conducting usability tests, I've heard of this contradictory behavior from other researchers. Such occurrences have led many to discount the collection of satisfaction data altogether. In fact I've often heard

SEQ

by Jeff Sauro, PhD and Jim Lewis, PhD | September 1, 2020

Browse Content by Topic

What Do You Gain from Larger-Sample Usability Tests?

By Jeff Sauro, PhD and Jim Lewis, PhD . September 1, 2020 . Tags: Sample Sizes, Formative, Summative, SEQ, Rating Scale, Sample Size

Sample Size Recommendations for Benchmark Studies

By Jeff Sauro, PhD and Jim Lewis, PhD . June 23, 2020 . Tags: Sample Sizes, SEQ, Sample Size, SUS

Approximating Task Completion When You Can’t Observe Users

By Jeff Sauro, PhD and Jim Lewis, PhD . April 14, 2020 . Tags: Unmoderated Research, SEQ, SUS

Cultural Effects on Rating Scales

By Jeff Sauro, PhD . February 4, 2020 . Tags: LTR, Satisfaction, SEQ

Using Task Ease (SEQ) to Predict Completion Rates and Times

By Jeff Sauro, PhD . October 30, 2018 . Tags: Validity, SEQ, Questionnaire

10 Things To Know About The Single Ease Question (SEQ)

By Jeff Sauro, PhD . October 30, 2012 . Tags: SEQ, Rating Scale

Do Users Fail A Task And Still Rate It As Easy?

By Jeff Sauro, PhD . October 9, 2009 . Tags: SEQ, Usability Metrics, Rating Scale, Questionnaire