Surveys are one of the most popular methods in applied research. While many have argued that surveys are overused, it’s hard to believe that surveys have no place in multi-method UX research.
When conducting survey-based research, you’ll often encounter the terms census, poll, and questionnaire used in conjunction with—and often interchangeably with—the term survey. But are they really the same? In this article, we describe their differences.
Census
A census is the process of gathering information about every member of a population. By definition, the goal is full enumeration, never sampling. Census is the word used when counting all the people in a country or region. The practice occurred in ancient Egypt, Rome, and Babylonia as much as 6,000 years ago. In the United States, a census is conducted every ten years (as outlined in the Constitution—see Figure 1). But a census can involve more than just counting people. For example, there’s an international census of agriculture.
Figure 1: Image from the U.S. Constitution establishing the national census: “The actual enumeration shall be made within three years after the first meeting of the Congress of the United States, and within every subsequent term of ten years, in such manner as they shall by law direct.”
In applied research, the most common usage of a census is not to conduct one but to use existing censuses provided by governments to create census-matching proportional probability samples—samples that are proportionally representative of the population with respect to variables such as geographic location and age.
Poll
In modern usage, poll is strongly associated with elections; for example, you go to a polling station to vote. In thirteenth-century English, it meant “a head.” At that time, a common process for local elections was for people to congregate in groups representing their choice as someone counted the heads in each group.
In applied research, polling usually refers to an opinion poll that gauges the attitudes of a population via sampling from that population. The first known opinion poll was taken of prospective voters in the U.S. presidential election campaign of 1824. Even today, polls are most associated with assessing political attitudes, such as the preference for a presidential candidate. Two of the most famous founders of opinion polling companies in the twentieth century are George Gallup and Elmo Roper, whose methods are still used today.
Some definitions of a poll specify that its distinguishing characteristic is that it asks just one question. Although modern polls often have a single core question (e.g., voting preference), they can include more than one question (e.g., likelihood to vote), making them increasingly indistinguishable from short surveys (especially when they contain demographic questions). Polls can best be thought of as a special type of short survey usually focusing on some preference or choice (Figure 2).
Figure 2: Example of a two-question opinion poll (created in MUIQ®).
Survey
The word survey refers to data collection methods that can include questionnaires or polls. The four primary methods used to conduct surveys are in-person, phone, mail, and online. Researchers use these four survey mechanisms to conduct censuses, so it’s common to distinguish surveys and censuses based on whether the goal is to fully enumerate a population (census) or to sample from it (survey). There are many different sampling strategies for surveys, each with its strengths and weaknesses.
The terms survey and questionnaire are often used interchangeably (perhaps we are guilty). For example, we’ve heard researchers refer to a SUS Survey or a Net Promoter Survey. However, both the SUS and NPS are, technically, questionnaires that can be included in a survey. Any given survey can include multiple questionnaires and additional questions (such as demographic or open-ended questions).
Questionnaire
A questionnaire is the set of questions asked of a participant in surveys and censuses. The first questionnaires appeared in the mid–eighteenth century (e.g., the “Milles” questionnaire, which contained 120 questions about the history of the parishes of the diocese of Exeter). UX questionnaires often contain groups of items to measure constructs such as satisfaction, usability, loyalty, or brand attitude.
A standardized questionnaire is a set of questions that has gone through the process of psychometric validation. This means someone has spent a lot of time going through dozens or hundreds of possible questions and winnowed the set down to the ones that are most reliable, valid, and sensitive. Usually, this process is done by having a representative group of participants answer the candidate questions for a diverse set of interfaces. Examples of standardized UX questionnaires include the SUPR-Q®, SUS (see Figure 3), PSSUQ/CSUQ, UX-Lite™, SASSI, and NASA TLX (and many more).
Some standardized questionnaires from other fields are the MMPI (clinical psychology and psychiatry), Wechsler Intelligence Scale for Children (psychology and education), and the Strong Interest Inventory (career assessment).
Figure 3: Example of a standardized UX questionnaire, the System Usability Scale (SUS) (created in MUIQ).
Summary
In this article, we’ve discussed similarities and differences between four research terms: census, poll, questionnaire, and survey.
Census: A census refers to methods in which characteristics of entire populations are fully enumerated, as opposed to sampling from a population. A census can include any type of question. and historically the goal of a census is to count members of a population for the governmental purposes of representation, conscription, and taxation.
Poll: Polls are strongly associated with elections (e.g., exit polls) and collecting consumer opinions (e.g., product preference polls). The defining characteristic of a poll is its short length. Some polls may contain a single question, but most have more than one (though not much more). Think of polls as special types of short surveys focused on preference or choice.
Survey: Surveys differ from censuses in their use of sampling from a population. Surveys do not attempt to collect data for every member of the population. They differ from polls primarily in their longer length (number of questions). Methods for conducting surveys include in-person, telephone, mail, and online.
Questionnaire: Questionnaires are the actual sets of questions given to respondents, as opposed to the method of delivery (census, poll, or survey). A special type of questionnaire is the standardized questionnaire, which have gone through a process of psychometric qualification that includes assessment of reliability, validity, and sensitivity.
These terms are all related to the process of collecting information from people using structured forms, as opposed to free-form interviews. The first three (census, poll, and survey) are methods that differ in their use or nonuse of sampling (census vs. survey) and length (poll vs. survey). All three methods are used to present questionnaires to respondents using any of the typical data collection methods—in-person, telephone, mail, or online.
