In 1906 Vilfredo Pareto, an Italian economist, observed that wealth was unequally distributed in Italy. He noted that 80% of the land and wealth was owned by 20% of the people.
A similar relationship can be observed in the wealth and income across most countries. A minority of the population tends to generate the majority of the income and controls most of the wealth. For example, the top 20% of earners make 51% of all income in the US[pdf].
But why should we care about early 20th century Italian economics and the distribution of wealth?
It turns out what Pareto observed isn’t limited to wealth and income.
In fact, Pareto would probably just be an even lesser known historical figure if it wasn’t for Joseph Juran, a pioneer in Quality Management. Juran developed the idea of “the vital few and the trivial many.” He saw this pattern repeatedly in business and gave it the name of the Pareto Principle. It’s often simply referred to as the 80/20 rule.
The principle is often generalized to mean that roughly 20% of the effort will generate 80% of the results. It’s often the case that 20% of customers generate 80% or more of revenue for a company. Or 20% of customers demand 80% of the new feature requests. However, the percentages do not have to be 80/20, nor do they always equal 100%. In many datasets I’ve analyzed, the percentages are even more lop-sided. I’ve often seen less than 10% of customers accounting for 90% of revenue, or 1% account for 50%.
Here are some more examples of the Pareto Principle
- Microsoft noted that by fixing the top 20% of the most reported bugs, 80% of the errors and crashes would be eliminated
- The top 10% of cell phone users consume 90% of wireless bandwidth.
- The top 20% of U.S, taxpayers pay 68% of all taxes[pdf]
Here are three examples of how the Pareto Principle applies to user research.
What Users Do: Top Task Analysis
Most websites support hundreds and even thousands of tasks. Even the biggest web team will get overwhelmed trying to ensure all these tasks are both functioning and usable. It turns out, though, that just a critical few tasks account for the majority of what your visitors come to websites to do. The Pareto Principle is therefore used to identify the top tasks users perform on a website to determine where the web team needs to concentrate its efforts.
Gerry McGovern famously calls this the “long neck” of task frequency and provides many examples in his book the Stranger’s Long Neck on how the trivial many tasks can interfere with the critical few. Gerry presented at the Lean UX Denver conference on September 20th 2012.
For example, we asked 222 potential users of a large informational website to pick the five most important things they want to accomplish out of 94 possible tasks. You can see Gerry’s “long neck” in the graph of the votes below. The top 15(16%) tasks account for 41% of all votes. Those are the vital few tasks that users come to this website to do. We use Gerry’s approach in identifying which tasks to administer in usability tests, then measure and improve.
Figure 1: Of the 94 tasks presented to users, just 15 (16%) account for 41% of all votes.
What one thing would you improve?
We usually ask users in our usability tests what one thing they’d improve about a website or software. When we convert these open-ended comments into categories and graph their frequency, we also see a Pareto distribution. In the example below of what users would fix, two issues account for 50% of all comments and five account for 75%.
Figure 2: When asked what users would fix on a website, two issues account for 50% of all comments and five account for 75%.
Usability Problem Frequency
Even in a formative usability test where we’re looking to find and fix usability problems we see the vital few and trivial many. When we log which user encounters which problem in a user-by-problem matrix we also see a Pareto distribution. The graph below shows the problems users encountered while trying to rent a car on the Enterprise website. In total, there were 33 unique issues encountered a total of 181 times. Two issues affected most users and then a long-tail of problems impacted only a single user. Notice that just nine problems (27%) account for 72% of all poor interactions.
Figure 3: Of the 33 usability problems identified across 50 users, fixing nine of them would address 72% of all poor interactions.
There’s a finite amount of resources to focus on finding and fixing issues. Don’t worry too much about finding the perfect 80/20 balance. Instead, look to find where a small number of items account for a disproportionate amount of results. An effective strategy is to separate these vital few from the trivial many to improve the user experience—whether it be usability problems, feature requests, calls in to support, software bugs or revenue.
|UX Measurement Boot Camp : Three Days of Intensive Training on UX Methods, Metrics and Measurement Aug. 7th-9th 2019|