All Blogs

While testing with five users might reveal 85% of problems that impact 31% of users (given a set of tasks and user-type), it doesn't mean you're finding 85% of the critical problems.  Are severe usability problems likely to occur more frequently, less frequently or is problem severity independent of frequency? The data on this is mixed. The paper from Bob Virzi in 1992 showed there

Read More

I'm always gathering and looking at data. One consequence of this is having to reconcile conflicting data-points—say data from users who express different perspectives on an issue. For example, one of my articles was recently tweeted with the note:  "any website with the name usability in it should let you know you're clicking on a PDF." Few things slow the web-browsing experience down more than

Read More

Was a task difficult or easy to complete? Performance metrics are important to collect when improving usability but perception matters just as much. Asking a user to respond to a questionnaire immediately after attempting a task provides a simple and reliable way of measuring task-performance satisfaction. Questionnaires administered at the end of a test such as SUS, measure perception satisfaction. There are numerous questionnaires to

Read More

Asking questions immediately after a user attempts a task compliments task-performance data such as task times and completion rates. Post-task satisfaction data is a bit different than the questionnaires asked after a usability test (such as the SUS).  There is a strong correlation (r > .6) between post-task ratings and post-test ratings. Knowing one can predict about 36% of the other[pdf]. However, even this relatively

Read More

Most usability testing involves finding and fixing problems as part of an iterative design process to make an interface more usable. It is typically called a Formative Usability Evaluation. In contrast, a Summative Usability Evaluation describes the current usability of an interface—as measured by things like task times, completion rates and satisfaction scores. Summative tells you how usable an interface is and formative tells you

Read More

There has been a bit of an explosion in remote usability testing tools of late. Some of these companies also offer to recruit and provide you with users who are paid to take your usability test (Usertesting.com, Feedback Army and EasyUsability to name a few). These services allow for quick (almost immediate) feedback from users. Data is gathered quickly via the internet as users are

Read More

Let's imagine you are testing five users as part of an iterative testing approach to find and fix problems. During the test only one user encounters a problem with logging in. To fix this particular problem would take a lot of effort and the small sample size is met with skepticism from the overburdened and overcommitted development team. They say "We really don't know whether

Read More

In a usability test you typically collect some type of performance data: task times, completion rates and perhaps errors or conversion rates. It is also a good idea to use some type of questionnaire which measures the perceived ease-of-use of an interface. This can be done immediately after a task using a few questions (post-task questionnaires). It can also be done after the usability testing

Read More

The System Usability Scale (SUS) is the most popular standardized usability questionnaire. SUS was developed about 20 years ago at Digital Equipment Corporation by John Brooke. It's popular for two reasons: it's free and short (at only 10 questions). The process of taking a set of ordinary questions and making it into a psychometrically valid and reliable "standardized" questionnaire essentially involves having many users answer

Read More