All Articles

Surveys are an essential method for collecting data. But like all research methods, surveys have their limitations. Unless the survey is administered by a facilitator, a respondent has only the survey's instructions, questions, and response options for guidance. Earlier we wrote about ways to improve the clarity of your questions and certain words to watch for when writing questions. But, despite your best efforts, inevitably

Read More

The food delivery market has been growing significantly over the past five years. That growth exploded in the United States from $17 billion in 2018 to $26 billion in 2020 (partly due to COVID-19). This market is highly competitive and has ultra-thin profit margins. Technologies such as route optimization enable faster and cheaper delivery, but dissatisfied customers can easily switch providers. Service providers that don’t

Read More

Like in all research methods, many things can go wrong in surveys, from problems with sampling to mistakes in analysis. To draw valid conclusions from your survey, you need accurate responses. But participants may provide inaccurate information. They could forget the answers to questions or just answer questions incorrectly. One common reason respondents answer survey questions incorrectly is that they misinterpret the question. Understanding the

Read More

When thinking about user experiences with websites or software, what is the difference between capabilities and functions? Is there any difference at all? In software engineering, a function is code that takes inputs, processes them, and produces outputs (such as a math function). The word capability doesn’t have a formal definition, but it most often appears in the phrase Capability Maturity Model, a formal model

Read More

Post-mortems and retrospective accounts are valuable for understanding what went well and what went poorly. This applies not only to investigations of critical events, such as crimes and plane accidents, but also to experiences with products and services. But the usefulness of people's recollections of events and experiences rests on the accuracy of their memories. (Measuring the perception of experiences might be just as valuable

Read More

Sample size estimation is a critical step in research planning. It can also seem like a mysterious and at times controversial process. But sample size estimation, when done correctly, is based mostly on math, not magic. The challenge is that the math can get complex, so it becomes easier to defer to simple rules or a one-size-fits-all number. Before 2016, there was no well-defined method

Read More

In UX research, both studies and surveys contain a lot of questions. Getting those questions right can go a long way in improving the clarity and quality of the findings. For example, we’ve recently written about how to make survey questions clearer. And while there are many stories of how the change of a single word in a survey question can lead to different results

Read More

The Net Promoter Score (NPS) is widely used by organizations. It’s often used to make high-stakes decisions on whether a brand, product, or service has improved or declined. Net Promoter Scores are often tracked on dashboards, and any changes (for better or worse) can have significant consequences: adding or removing features, redirecting budgets, even impacting employee bonuses. Random sampling error, however, can often explain many

Read More

The first questionnaires appeared in the mid–18th century (e.g., the “Milles” questionnaire). Scientific surveys have been around for almost a hundred years. Consequently, there are many sources of advice on how to make surveys better. The heart of each survey is the questions asked of respondents. Writing good survey questions involves many of the principles of good writing, in addition to survey-specific guidelines and advice.

Read More

When we wrote Quantifying the User Experience, we put confidence intervals before tests of statistical significance. We generally find fluency with confidence intervals to be easier to achieve and of more value than with formal hypothesis testing. We also teach confidence intervals in our workshops on statistical methods. Most people, even non-researchers, have been exposed to the concept of margins of error—political polls include them.

Read More