UX Webinar – Rating Scales Best Practices: The conventional wisdom may not always be wise

UX Webinar

Rating Scales Best Practices: The conventional wisdom may not always be wise

 

Speakers:

Jeff Sauro, PhD, and Jim Lewis, PhD

 

Event Date:

Wednesday, April 29th, 2020

10:00 AM MDT / 9:00 AM PDT

 

Register Here!

 

Rating scales are ubiquitous.  But so it seems are “best practices” on what you should and shouldn’t do with rating scales. It can be hard to separate scientific and research-based findings versus just conventions that are followed or myths that are perpetuated.  In this webinar Jeff Sauro, PhD and Jim Lewis, PhD will address several common questions about rating scale usage for UX research and provide guidance based on the research.

Topics covered will include:

  • How many points should you use?
  • Are three points enough?
  • Should you always label your rating scale points?
  • How much does country/region matter?
  • Should you have a neutral point and should you label it?
  • Should you use a mix of positive and negative items in a questionnaire?

Jeff and Jim will review the results of dozens of studies conducted by MeasuringU and summarize the published literature to highlight areas you should be concerned about and other areas where differences in rating scale formats don’t matter much.

Register Here!

UX Webinar – Rating Scales : How Much Do Format Changes Affect Responses?

UX Webinar

Rating Scales : How much do format changes affect responses?

 

Speakers:

Jeff Sauro, PhD, and Jim Lewis, PhD

 

Event Date:

Wednesday, September 16th, 2020

10:00 AM MDT / 9:00 AM PDT

 

Register Here!

 

Rating scales are ubiquitous and come in all sorts of varieties…and so do opinions on what works and what doesn’t.

In our earlier webinar we reviewed some of the “best practices” around rating scales and helped separate research-based findings from myths and anecdotes.  We covered a lot of topics and addressed several questions about different presentation styles. But we didn’t have the data to offer an informed opinion on all questions we received in the webinar. Now we have data and opinions.

In this webinar Jeff Sauro, PhD and Jim Lewis, PhD will review how different rating scale presentation formats did or did not affect response patterns in studies conducted by MeasuringU and in the published literature.  The different formats we’ll cover include:

  • Stars vs Numbers
  • Emojis vs Numbers
  • Scales with Negative Numbers vs All Positive
  • Sliders vs Numbers

Jeff and Jim will summarize the key findings and relevant published literature to highlight areas you should be concerned about and other areas where differences in rating scale formats don’t matter much.

Register Here!

UX Webinar: The Essentials of UX Benchmarking & Its 3 Biggest Challenges

At MeasuringU we’ve conducted hundreds of benchmarks on websites, software, apps and hardware. In this webinar Jeff Sauro & Jim Lewis will cover the essentials of benchmarking the User Experience and what we’ve seen as the biggest challenges and how to overcome them.

UX Webinar

The Essentials of UX Benchmarking & Its 3 Biggest Challenges

Speakers: Jeff Sauro, PhD, and Jim Lewis, PhD

 

Event Date: Wednesday, March 18th, 2020

10:00 AM MDT / 9:00 AM PDT

Register Here!

At MeasuringU we’ve conducted hundreds of benchmarks on websites, software, apps and hardware. Benchmark studies are a great way to measure an experience, but often have their own challenges.

In this webinar Jeff Sauro & Jim Lewis will cover the essentials of benchmarking the User Experience and what we’ve seen as the biggest challenges and how to overcome them. We’ll cover:

  • What UX Benchmarking is and why and when to conduct a benchmark
  • What metrics to collect
  • Deciding between a Comparative or a Stand-Alone Benchmark
  • Finding the Right Sample Size

UX Book Club Denver – Jeff Sauro, “Benchmarking the User Experience”

UX Book Club Denver

Benchmarking the User Experience

Presentation by author – Jeff Sauro, PhD

 

Event Date: Wednesday, November 28th, 2018

Location: MeasuringU, 3300 E 1st Ave, Suite 370, Denver, CO 80202

7:00 – 9:00 PM

Event Website

 

Join us at MeasuringU for the UX Book Club Denver event on Wednesday, November 28th at 7:00 PM! At this event, Dr. Jeff Sauro presents an overview of his new book, Benchmarking the User Experience, and a talk on UX benchmarking.

Additionally, be ready for a night of networking and mingling with the accompaniment of drinks and appetizers.

You may find free street parking located around the block of our building (3300 E 1st Ave, Suite 370, Denver, CO 80202).

 

Benchmarking the User Experience

 

Benchmarking the User Experience is a practical book about how to measure the user experience of websites, software, mobile apps, products, or just about anything people use. This book is for UX researchers, designers, product owners, or anyone who has a vested interest in improving the experience of websites and products.

 

In this book, Jeff Sauro, Ph.D. uses practical examples to illustrate what benchmarking is and how you can use this technique to measure users’ experiences. Dr. Sauro uses many examples of studies from his company, MeasuringU, to give constructive information that will help you formulate and/or understand your benchmarking studies, which will ultimately help you understand your audience and customers.

 

Making Better Design Decisions With Data

Jeff Sauro, Ph.D., author of “Quantifying the User Experience,” will be a featured speaker at Rosenfeld Media’s virtual conference on “The Business Case for Design.”

He will be speaking from 10:00-10:50 AM (MST).

Dr. Sauro will be discussing topics that include: how to apply scientific thinking toward several design scenarios; how to form a testable hypothesis; ways to operationalize a study with the right tasks, metrics and methods; the importance of randomization to minimize nuisance variables; and analyzing data to differentiate real change from random noise.

For more information on the conference or to purchase tickets, visit https://rosenfeldmedia.com/business-case-for-design/

Get 15% off ticket prices by entering the code: SAURO15!

UX Measurement Bootcamp 2020

4 Days of Intensive Training on UX Methods, Metrics and Measurement

Virtual for 2020, 5 hours/day

 


 

Event Dates: October 13 – October 16, 2020

Event Location: Online from 8am – 1pm (Pacific Time)

 

 


 

This is our 8th Annual Bootcamp, tailored for online learning

 

Each year we’ve had a great group of participants from dozens of companies including:

Aetna, Amazon, American Airlines, Amica Insurance, Autodesk, BlackRock, Boeing, Charter Communications, Cisco, Citigroup, Google, IBM, John Deere, Rockwell Automation, Samsung, Spotify, Target, The Home Depot, Workday.

 


 

Participants receive:

●    a User Experience Research Certificate

●     2.5 Continuing Education Credits from the University of Denver.

 

 

[My bosses] are thrilled with the possibilities that this opened up for us in terms of quantifying user feedback and behavior.

You gave me the tools and skills to give more credibility to my work…I am very grateful.

 


 

Course Details

This year the UX Measurement Bootcamp will be held online. Class times will be modified to work for online delivery.

Jeff Sauro, PhD & Jim Lewis, PhD will be the instructors.

The courses will be a mix of (1) lecture material and (2) practice learning and applying the methods (adapted for online delivery).

The online course cost is $2700 per participant. This is ~20% lower than our in-person price of $3,300.

 

This includes:

●     Course Materials (valued at over $700)

●     Software Licenses (downloadable Excel calculators)

●     Up to 100 US-based participants for your study

●      Certificate and Continuing Education Credit from the University of Denver

 

Contact us if you have more than 5 attendees or if you are interested in a custom bootcamp for your company.

 

 

I thought the bootcamp was extremely helpful. Jeff did a great job of teaching a lot of material in a short amount of time, and I am already using these ideas in my work.

We had a lot of fun and success in applying what I learned in the bootcamp to my work.

 


 

Topics Covered

●     An Overview of UX Methods & Metrics

●     Determining Customers’ Top Tasks

●     Making the Most of Unmoderated Usability Testing

●     UX Benchmarking

●     Testing the Information Architecture / Navigation of Websites

●     Card Sorting and Tree Testing

●     How to Present Quantitative UX Data

●     Best Practices for Survey Design and Analysis

●     Sample Size Calculations

●     Confidence Intervals & Statistical Comparisons

●     Making the Most of Small Sample Sizes

 

 

 

 


 

Bootcamp Course Schedule (subject to change)

The UX Measurement Bootcamp includes interactive lectures, visuals, witty jokes, and practice exercises, along with hands-on workshops focusing on more detailed topics. Schedules and topics may change based on participants’ backgrounds and current needs.

 

Day 1

Tuesday

Introduction to UX Methods and Metrics
A 100 year history of usability in pictures, people and events followed by a discussion of how usability is defined and measured across dozens of organizations. We will review common methods for measuring usability and user experiences. This will follow Chapters 1-2 in Quantifying the User Experience.
Top Tasks Analysis
It’s difficult to do everything well all the time. One of the biggest problems in interactions we see in software and websites is that developers and designers think users are doing one thing when in fact users are doing something else. The top task analysis is a simple, yet powerful way of understanding what critical few tasks tend to affect the bulk of the user experience.
Findability & Tree Testing
Despite improvements in search technology, browsing is still a dominant strategy used by most users. At the heart of findability problems is a navigation with a confusing structure and poorly chosen category labels. We will discuss how to conduct a tree-test to isolate and generate a valid quantitative baseline measure of findability. We will explore strategies for improving findability through card sorting and cross-linking and then comparing a new navigation structure to the baseline.
Study Development
Introducing our framework for developing UX studies, then breaking up into groups for an exercise in study development.
Benchmark Usability Testing
We will review a case study for two online rental car company websites using the standard usability metrics to understand how to obtain a valid usability benchmark and set the foundation for statistical decision making with UX metrics.
Introduction to MUIQ (Team Formation & Study Goals)
During the bootcamp, attendees will work in teams to conduct an unmoderated usability study with our MUIQ tool. The first day ends with introduction of the tool and formation of teams.

Day 2

Wednesday

Confidence Intervals
Sampling users of any size involves sampling error. Confidence intervals provide the level of precision around estimates of findability, completion rates, likelihood to recommend, and any metric you collect from a sample of users. We will cover both the concepts of confidence intervals and normal theory and how to apply this practice on any sample from 2 to 200,000 users. This will follow Chapter 3 in Quantifying the User Experience.
Project: Design & Deployment
In this two-hour workshop, we’ll give you and your team hands-on instruction in setting up and preparing to analyze an unmoderated study (of your choosing). Once ready, it will be launched to collect real data from targeted participants.

Day 3

Thursday

Hypothesis Testing & Statistical Significance
With everyone wide awake on caffeine, we will get into both the theoretical and practical applications of testing hypotheses in the user experience. We will cover the basics of statistical testing and get practice determining if two websites or designs are statistically different. Participants will receive a comprehensive Excel calculator and guide to walk them through how to conduct statistical tests for the most common situations in applied user research. This will follow Chapters 5 and 10 in Quantifying the User Experience.
Project: Analyze Data & Prepare Presentations
Breaking again into project teams, you’ll get help in preparing your analyses, conducting statistical calculations, and presenting your findings.
Observing & Coding Usability Problems
At the heart of usability testing, whether it be the do-it-yourself kind or the more industrial strength kind, is documenting, diagnosing and reducing usability problems. Following up from the early quantitative benchmark data we will watch videos of users attempting to find the cost of dog grooming,  documenting and discussing the usability problems.

Day 4

Friday

Power and Sample Size Calculations
How many users do you test? It’s one of the most common and confusing questions in applied user research. Instead of relying on vague rules of thumb, you’ll learn how to compute sample sizes for making comparisons, discovering problems in usability tests, and for survey research.
Project: Finish Analysis and Report Preparation
Breaking again into project teams, you’ll complete your analyses, statistical calculations, and presentations.
Presentations & Ending
We finish with team representatives giving their presentations, then wrapping up with a final Q&A.

 

 

 

Thank you for the wonderful training…loved it!

I love, love, loved the class! It was by far the best training that I’ve attended in my career (and I have been working for a very long time).

Jeff is a phenomenal speaker and teacher. He makes very difficult (and let’s face it, pretty dry) stuff fun to learn. Thanks for putting this on!

 

 


 

 

Course Faculty

 

Jeff Sauro Jeff Sauro will be the primary instructor for the bootcamp. He has taught UX research methods classes for companies including Autodesk, Cisco, Dropbox and Salesforce. He is a regular speaker at UX conferences including UXPA and CHI and is an adjunct instructor at the University of Denver.
Jim Lewis Jim Lewis will be the assistant instructor for the bootcamp. He spent 40 years as a UX researcher/consultant at IBM. While at IBM, he collaborated with Jeff on a number of projects, including co-authoring numerous books and conference papers (two of which were nominated for Best of CHI). He has published almost 100 research articles and papers, is on 91 US patents, and in 2020 joined MeasuringU as Distinguished User Experience Researcher.

 

 


 

 

Substitution and Carry-Over Policy

Due to the customized nature of this training, we cannot offer refunds but we accept substitutions. Please contact us as soon as you know you will have a substitute. We may also carry over your registration for the following year.

 

 


 

 

UX Measurement Bootcamp 2019

3 Days of Intensive Training on UX Methods, Metrics and Measurement


 

Event Date: August 7 – 9, 2019

Event Location: Denver, Colorado

 

 


 

This is our 7th Annual Bootcamp.

 

Each year we’ve had a great group of participants from dozens of companies including:

Aetna, Amazon, American Airlines, Amica Insurance, Autodesk, BlackRock, Boeing, Charter Communications, Cisco, Citigroup, Google, IBM, John Deere, Rockwell Automation, Samsung, Spotify, Target, The Home Depot, Workday.

 


 

Participants receive:

●    a User Experience Research Certificate

●     2.5 Continuing Education Credits from the University of Denver.

[My bosses] are thrilled with the possibilities that this opened up for us in terms of quantifying user feedback and behavior.

You gave me the tools and skills to give more credibility to my work…I am very grateful.


 

Course Details

The early registration discounted cost is $3,000 US per participant for all three days (through July 1st). The regular price is $3,300.

 

This includes:

●     Daily Lunch and Two Daily Breaks

●     Course Books, Binders and Materials (valued at over $700)

●     Software Licenses (downloadable Excel calculators)

●     Up to 100 US-based participants for your study

●     Internet Access

●     Certificate and Continuing Education Credit from the University of Denver

●     Thursday Night Cocktail Reception

 

Classes will run from approx. 9am-5pm daily with lunch and a morning and afternoon break. Jeff Sauro will be the instructor. The courses will be a mix of lecture material and hands on practice learning and applying the methods.

All classes will be held on the University of Denver Campus.

 

 

 

I thought the bootcamp was extremely helpful. Jeff did a great job of teaching a lot of material in a short amount of time, and I am already using these ideas in my work.

We had a lot of fun and success in applying what I learned in the bootcamp to my work.


 

Topics Covered

●     An Overview of UX Methods & Metrics

●     Determining Customers’ Top Tasks

●     Making the Most of Unmoderated Usability Testing

●     UX Benchmarking

●     Testing the Information Architecture / Navigation of Websites

●     Card Sorting and Tree Testing

●     How to Present Quantitative UX Data

●     Best Practices for Survey Design and Analysis

●     Sample Size Calculations

●     Confidence Intervals & Statistical Comparisons

●     Making the Most of Small Sample Sizes

 

 

 

 

 


 

Bootcamp Course Schedule (subject to change)

The UX Measurement Bootcamp will involve mornings with interactive lectures, visuals, witty jokes, and practice exercise. Afternoons will involve more hands-on workshops focusing on more detailed topics. Schedules and topics may change based on participants’ backgrounds and current needs.

Day 1

Wednesday

Introduction to Usability & User Measurement
A 100 year history of usability in pictures, people and events followed by a discussion of how usability is defined and measured across dozens of organizations. We will review common methods for measuring usability. This will follow Chapters 1-2 in Quantifying the User Experience.
Top Tasks Analysis
It’s difficult to do everything well all the time. One of the biggest problems in interactions we see in software and websites is that developers and designers think users are doing one thing when in fact users are doing something else. The top task analysis is a simple, yet powerful way of understanding what critical few tasks tend to affect the bulk of the user experience.
Benchmark Usability Testing
We will review a case study for two online rental car company websites using the standard usability metrics to understand how to obtain a valid usability benchmark and set the foundation for statistical decision making with UX metrics.
Findability & Tree Testing
Despite improvements in search technology, browsing is still a dominant strategy used by most users. At the heart of findability problems is a navigation with a confusing structure and poorly chosen category labels. We will discuss how to conduct a tree-test to isolate and generate a valid quantitative baseline measure of findability. We will explore strategies for improving findability through card sorting and cross-linking and then comparing a new navigation structure to the baseline.
Confidence Intervals
Sampling users of any size involves sampling error. Confidence intervals provide the level of precision around estimates of findability, completion rates, likelihood to recommend, and any metric you collect from a sample of users. We will cover both the concepts of confidence intervals and normal theory and how to apply this practice on any sample from 2 to 200,000 users. This will follow Chapter 3 in Quantifying the User Experience.
Observing & Coding Usability Problems
At the heart of usability testing, whether it be the do-it-yourself kind or the more industrial strength kind, is documenting, diagnosing and reducing usability problems. Following up from the early quantitative benchmark data we will watch videos of users attempting to rent a car and document and discuss the usability problems.
PM Workshop: Conducting Usability Tests
After covering the different methods, now we will get hands on and break into groups to work through the core usability testing methods of moderated sessions, unmoderated tests and tree-testing.

Day 2

Thursday

Hypothesis Testing & Statistical Significance
With everyone wide awake on caffeine, we will get into both the theoretical and practical applications of testing hypotheses in the user experience. We will cover the basics of statistical testing and get practice determining if two websites or designs are statistically different. Participants will receive a comprehensive Excel calculator and guide to walk them through how to conduct statistical tests for the most common situations in applied user research. This will follow Chapters 5 and 10 in Quantifying the User Experience.
A/B Testing and Metric Comparisons
We will continue getting practice forming hypotheses and testing the differences in designs by going more in depth into conducting A/B tests and determining if two designs are statistically significant at small and very large sample sizes. We will focus on balancing practical significance with statistical significance through a series of examples and exercises.
Project Design & Deployment
In this half-day workshop, we’ll give you hands-on instruction in going through setting up and preparing to analyze an unmoderated study (of your choosing) and prepare to launch it to collect real data from targeted participants.

Day 3

Friday

Power and Sample Size Calculations
How many users do you test? It’s one of the most common and confusing questions in applied user research. Instead of relying on vague rules of thumb, you’ll learn how to compute sample sizes for making comparisons, discovering problems in usability tests, and for survey research.
Findability & Usability Case Studies
We will walk through a series of scenarios to bring together the methods learned earlier and apply research design, sample size calculations, metric selection and statistical comparisons. Working in teams, participants will design and conduct an unmoderated or findability study for analysis and discussion.
Final Analysis & Project Presentations
You’ll get help in preparing your analysis and statistical calculations then present your findings.

 

 

Thank you for the wonderful training…loved it!

I love, love, loved the class! It was by far the best training that I’ve attended in my career (and I have been working for a very long time).

Jeff is a phenomenal speaker and teacher. He makes very difficult (and let’s face it, pretty dry) stuff fun to learn. Thanks for putting this on!

 

 


 

 

Course Faculty

 

Jeff Sauro Jeff Sauro will be the primary instructor for the bootcamp. He has taught UX research methods classes for companies including Autodesk, Cisco, Dropbox and Salesforce. He is a regular speaker at UX conferences including UXPA and CHI and is an adjunct instructor at the University of Denver.

 

 


 

 

Accommodations

Hyatt Place Denver Cherry Creek will be offering discounted rates for UX Bootcamp attendees. The group rate is $139 for singles and $149 for doubles per night.

 

To book your reservation, please visit:
https://www.hyatt.com/en-US/hotel/colorado/hyatt-place-denver-cherry-creek/denzc?corp_id=G-UXBC

You can also book a reservation by calling 888-492-8847 and reference UX Bootcamp.

 

If you need any assistance in booking a room, please contact Abigail Young at abigail.young@hyatt.com or 303-389-6075.

For additional nearby hotel accommodations options, please visit:
https://www.du.edu/communityvisitors/visitors/hotels.html

 

 


 

 

Substitution and Carry-Over Policy

Due to the customized nature of this training, we cannot offer refunds but we accept substitutions. Please contact us as soon as you know you will have a substitute. We may also carry over your registration for the following year.

 

 

CHI 2017

MeasuringU official presentation date TBD

Jeff and the MeasuringU Team will present:

From Snake-Oil to Science: Measuring UX Maturity

Colorado Convention Center, Denver, CO | May 10, 2017 | 4:30pm

See more program information and register at the CHI 2017 website