At MeasuringU®, we don’t just use UX metrics—we create them. But what have we created, and what have we just used or extended? Across our combined careers, we (Jeff and Jim) have published 16 psychometrically qualified UX metrics (both creating original and modifying existing questionnaires) plus a method for combining prototypical usability metrics, and we
The word “standardized” conjures memories of high-stakes tests. In the context of UX research, when we talk of standardization, we’re often referring to standardized questionnaires. Standardized questionnaires have gone through the process of psychometric validation. That means the items being used have gone through dozens or hundreds of possible variations, and the final versions are
What makes a measure valid? Good intentions? Because someone influential said to use it online? A measure is valid if it can be demonstrated that it measures what it is intended to measure, has the expected alignment of items with factors, and has the expected statistical relationships with other metrics. Its usage also depends on
The standard deviation is the most common way of measuring variability or “dispersion” in data. The more the data is dispersed, the more measures such as the mean will fluctuate from sample to sample. That means higher variability (higher standard deviations) requires larger sample sizes. But exactly how much do standard deviations—whether large or small—impact
Are customers more satisfied this quarter than last quarter? Do users trust the brand less this year than last year? Did the product changes result in more customers renewing their subscriptions? When UX researchers want to measure attitudes and intentions, they often ask respondents to complete multipoint rating scale items, which are then compared with
Sample size estimation is an important part of study planning. If the sample size is too small, the study will be underpowered, meaning it will be incapable of detecting sufficiently small differences as statistically significant. If the sample size is too large, the study will be inefficient and cost more than necessary. A critical component
Many topics about the design of rating scales can provoke strong opinions and heated debates. The arguments include whether or not scales should include a neutral response. Unlike rating scales with an even number of points (e.g., 4, 6, or 10), rating scales with an odd number of points (e.g., 5, 7, or 11) contain
How do you measure tech savviness? For several years (since 2015), we’ve been on a mission to develop a valid and practical measure. In our earlier articles, we have Reviewed the literature. We reviewed the literature on tech-savvy measures and found three key approaches to measuring tech-savviness by assessing (1) what a person knows, (2)
The standard deviation is the most common measure of variability. It’s less intuitive than measures of central tendency such as the mean, but it plays an essential role in analysis and sample size planning. The standard deviation is a key ingredient when building a confidence interval and can be easily computed from a sample of
In an earlier article, we described a pilot study from 2015 in which we investigated how to measure tech savviness. Building on the published literature, we generated candidate items that measured three aspects of tech savviness: what people know, what people do, and what people feel. In that pilot study, we assessed knowledge using a
