Topics
Topics
Restoring Confidence in Usability Results
Adding confidence intervals to completion rates in usability tests will temper both excessive skepticism and overstated usability findings. Confidence intervals make testing more efficient by quickly revealing unusable tasks with very small samples. Examples are detailed and downloadable calculators are available. Are you Lacking Confidence? You just finished a usability test. You had
Calculating Sample Size for Task Completion (Discrete-Binary Method)
One of the biggest and usually first concerns levied against any statistical measures of usability is that the number of users required to obtain “statistically significant” data is prohibitive. People reason that one cannot with any reasonable level of confidence employ quantitative methods to determining product usability. The reasoning continues something like this: “I have
Calculating Sample Size for Task Times (Continuous Method)
We already saw how a manageable sample of users can provide meaningful data for discrete-binary data like task completion. With continuous data like task times, the sample size can be even smaller. The continuous calculation is a bit more complicated and involves somewhat of a Catch-22. Most want to determine the sample size ahead of
Calculating A Sigma Level From Task Success
Often the most reported measures of usability is task success. Success rates can be converted into a sigma value by using the discrete-binary defect calculation: Proportion Unsuccessful= Defects/Opportunities Where opportunities are the total number of tasks and defects are the total number of unsuccessful tasks. This calculation provides a proportion that is equivalent to a
Current Usability Solutions Are Unpredictable
It’s a big step when User Centered Design methods are employed in a company to improve the usability of a product. It shouldn’t be the last step and often times it is. Many popular usability testing techniques are the right method to gather user data, however, their results alone will only scratch the surface of
What’s a Z-Score and Why Use it in Usability Testing?
The Power of Z A common statistical way of standardizing data on one scale so a comparison can take place is using a z-score. The z-score is like a common yard stick for all types of data. Each z-score corresponds to a point in a normal distribution and as such is sometimes called a normal
Measuring & Analyzing Task Times
Continuous Data Efficiency is one of the cardinal aspects of a product’s usability. The amount of time it takes for a user to complete a task is one of the best predictors of efficiency because it: Illustrates the slightest variability (in seconds) that are more difficult to detect in other measures like errors Is a
What is an Acceptable Level of Quality for Usability?
Usability measurements involve human performance and because human behavior is inherently error prone, reaching the goal of 6σ isn’t necessary to proclaim success. Manufacturing companies that are considered producing “high-quality” products are usually somewhere between 4σ and 5σ. The benchmarks and targets that we set in our tests will necessarily need to be more forgiving
The Importance of Task Order Randomizing during a Usability Test
Minimize Lurking Variables Getting Warmed Up Without task randomization, so-called lurking variables can taint your data–usually not enough that it’s devastating but often it’s noticeable. One lurking variable when analyzing task times is the user’s tendency to perform better on the later tasks and worse on the earlier tasks. It’s human nature: someone hands you
What’s the 1.5σ Shift and Does it Apply to Software Usability?
If you compare the sigma value on this site with other values published in most six sigma literature, it’s important to know that a 1.5 σ “shift” is usually added. For example, if you see a sigma value of 1.08σ on measuringusability.com and want to compare it to other sources then add 1.5. The resulting
Why 6σ is Not Limited to Manufacturing Processes
Six Sigma History and Overview Six Sigma was started in manufacturing for processes that are duplicated thousands and millions of times in something like the placement of a spot weld on sheet metal on a power turbine and being able to predict the failure rate of that part’s weld. If you can measure and control
How Do You Calculate a Z-Score/ Sigma Level?
The benefit of using a z-score in usability metrics was explained in “What’s a Z-Score and why use it in Usability Testing?” this article discusses different ways of calculating a z-score. The short answer is: It depends on your data and what you’re looking for. If you’ve encountered the z-score in a statistics book you