Often the most reported measures of usability is task success. Success rates can be converted into a sigma value by using the discrete-binary defect calculation:

Proportion Unsuccessful= Defects/Opportunities

Where opportunities are the total number of tasks and defects are the total number of unsuccessful tasks. This calculation provides a proportion that is equivalent to a success or failure rate. For example, if 143 total tasks were attempted and 123 were successful(20 defects 143 opportunities) the equation would look like:

20/143 = .1399
By expressing .1399 as a percentage you get an unsuccessful rate of 13.9%. By subtracting the unsuccessful proportion from 1 you’ll get approximately .86 or expressed in a percentage you could say that the task completion rate was 86%. Converting the proportion to a sigma value is a little more tricky.

Converting a Proportion to a Sigma Value

A sigma value is a description of how far a sample or point of data is away from its mean, expressed in standard deviations usually with the Greek letter σ or lower case s. A data point with a higher sigma value will have a higher standard deviation, meaning it is further away from the mean.

Standard deviations and sigma values assume your data is normally distributed from a continuous set of data. A normal curve has a mean of 0 and a standard deviation of 1. Success rates are Discrete-Binary (either a task was completed or it wasn’t) not continuous(like time, weight or temperature) which means that binary data is not normally distributed.

Have no fear, for every statistical problem a statistical solution usually exits. We can treat binary data like continuous data because of the Central Limit Theorem. It states that as a binary sample gets larger, its distribution approximates a continuous distribution.

Approximating the Proportion to a Normal Distribution

An easy way to exploit the Central Limit Theorem solution and obtain a sigma value from the task proportion is by using the Excel function NORMSINV. NORMSINV approximates the area under a normal bell-shaped curve. Take the unsuccessful proportion .1399–the proportion defective and insert it into the following equation in Excel.

=NORMSINV(1-.1399) Note: My sigma values DO NOT contain a 1.5σ adjustment or “shift.”