The Many Ways of Thinking Aloud

The Think Aloud method (TA) is one of the most distinguishing characteristics of usability testing. The method involves having participants speak their thoughts as they attempt tasks on an interface.

We often think of the TA method as a single method, but there are substantial variations in how it’s implemented. In an earlier article, we discussed the history of thinking aloud in UX research, from Sigmund Freud to Ericsson and Simon (1980 [PDF]).

We’ve been investigating the impacts of TA on UX metrics, problem discovery, and even how people view a page. Before we can adequately assess the impact of TA, we need to better understand how it’s currently implemented. This article covers the four common types of thinking aloud, adapted from our chapter in Handbook of Human Factors and Ergonomics (5^th ed.).

1.   Concurrent TA

Likely the most common approach for using Think Aloud is what’s come to be known as concurrent Think Aloud. The method was originally based loosely on Ericsson and Simon’s work, but in practice, it has evolved into something based more on speech communication.

In concurrent TA, a moderator explains the method of speaking thoughts as participants attempt tasks. Moderators then prompt for more utterances when participants wane in talking, using different cues to encourage participants to keep talking.

There are some variations in concurrent TA protocols:

• Inconsistency in explanations to participants about how to think aloud
• Possibility of giving participants a warm-up or practice task
• Styles of reminding participants to think aloud (e.g., spoken reminder to keep talking, acknowledgment tokens such as “mm-hm”)
• Intervals and frequency of prompting and intervention

Boren and Ramey (2000) noted that TA practice often does not conform to the theoretical basis most often cited for it (Ericsson & Simon, 1980). They suggested that a better basis would be speech communication theory, with clearly defined communicative roles for the participant (in the role of domain expert or valued customer, making the participant the primary speaker) and the usability practitioner (the learner or listener, thus a secondary speaker). Experimental comparisons of Boren and Ramey with Ericsson and Simon protocols show they tend to produce similar outcomes.

2.   Retrospective TA From Video

One of the concerns with concurrent TA is the possible impact of the verbalizations and interruptions—even subtle cues—on participant performance. While it would seem that having someone think aloud would increase task time, the data are mixed on whether task times with TA are actually longer. In fact, some researchers have reported faster task completion with TA. When TA takes longer, the moderator’s style of interrupting or probing may lengthen the time more than the participant’s speaking aloud.

An alternative approach that avoids any interruptions or probing is stimulated retrospective TA. Participants perform tasks silently, as if a moderator wasn’t present. At the end of the task, participants review recordings of their task performance with the moderator and provides thoughts on what they were doing. In addition to possible problems with post-hoc rationalization, this approach at least doubles the time needed to complete one task in an already-crammed usability study (likely making the cure worse than the disease).

Bowers and Snyder (1990) reported similar task performance and subjective measures for concurrent and retrospective TA, but participants provided different types of information as a function of TA style. Participants in the concurrent condition tended to provide procedural information, and participants in the retrospective condition tended to give explanations and design statements.

3.   Retrospective TA from Memory

Rather than viewing a video of their task performance, an alternative is to have a participant attempt a task first. At the end of the task, the participant reads the task description and recalls what happened out loud. This eliminates the time and tech needed to replay videos but relies heavily on participant memory. A study by McDonald, Zhao, and Edwards (2013) found that this approach produced more verbalizations that were relevant to usability analysis, despite the occurrence of a small number of less desirable utterances (hypothesizing, rationalizing, forgetting).

4.   Unmoderated Remote TA

A more recent addition to the Think Aloud repertoire has come with the proliferation of remote UX evaluation.

In unmoderated remote TA, participants receive instructions on how to think aloud, and are then they are asked to think aloud during a task. In many cases, there are instructional videos; in some cases, repeated digital prompts during the tasks to remind participants to continue talking. No remote moderator is present to prompt or probe.

Many platforms offer unmoderated remote TA, including our MUIQ^® platform, and this approach saw widespread adoption with the introduction of services such as Usertesting.com in the 2010s. One criticism of these services is that in order to have enough participants to think aloud, a panel of “professional testers” must be created, and participants are penalized for lackluster verbalizations, bringing into question the generalizability of data collected from this captive panel of trained participants.

Discussion

Just like there’s more than one type of usability test, there’s more than one way to implement the Think Aloud approach. Table 1 summarizes some key differences between the four approaches of Think Aloud that we covered: concurrent TA (with a moderator), two retrospective protocols (one with video and one from memory), and remote TA (no moderator). In future articles, we’ll explore additional details on the different approaches and how they may impact data.

Table 1: Properties of four TA protocols.