All Blogs

How long does it take users to complete a task? We really don't know. Instead we have to take our best guess from a sample of users. But if you had to pick a single number to summarize how long it would take typical users to complete a task from a usability test what would you report?  The mean? The median? The mode? Something else?

Read More

A user-interface problem is anything in the code or design that inhibits a user from completing an intended action. Usability testing is best done in the formative stages of development to find these UI problems and generate quantifiably better designs. Usability testing also has a summative role at the end of development as validation when bugs and functional gaps have been addressed.  Most UI problems

Read More

While it might seem like having users think aloud while they complete a task will increase the time on task—the literature is actually mixed. Some studies report no difference, some show longer task times for thinking aloud and others report faster task times for thinking aloud. It has been hypothesized that the reason users can perform faster while they think aloud is because this vocalization

Read More

While testing with five users might reveal 85% of problems that impact 31% of users (given a set of tasks and user-type), it doesn't mean you're finding 85% of the critical problems.  Are severe usability problems likely to occur more frequently, less frequently or is problem severity independent of frequency? The data on this is mixed. The paper from Bob Virzi in 1992 showed there

Read More

I'm always gathering and looking at data. One consequence of this is having to reconcile conflicting data-points—say data from users who express different perspectives on an issue. For example, one of my articles was recently tweeted with the note:  "any website with the name usability in it should let you know you're clicking on a PDF." Few things slow the web-browsing experience down more than

Read More

One question I get a lot is, Do you really only need to test with 5 users? There are a lot of strong opinions about the magic number 5 in usability testing and much has been written about it (e.g. see Lewis 2006PDF). As you can imagine there isn't a fixed number of users that will always be the right number (us quantitative folks love

Read More

Was a task difficult or easy to complete? Performance metrics are important to collect when improving usability but perception matters just as much. Asking a user to respond to a questionnaire immediately after attempting a task provides a simple and reliable way of measuring task-performance satisfaction. Questionnaires administered at the end of a test such as SUS, measure perception satisfaction. There are numerous questionnaires to

Read More

Asking questions immediately after a user attempts a task compliments task-performance data such as task times and completion rates. Post-task satisfaction data is a bit different than the questionnaires asked after a usability test (such as the SUS).  There is a strong correlation (r > .6) between post-task ratings and post-test ratings. Knowing one can predict about 36% of the other[pdf]. However, even this relatively

Read More