10 Double-Barreled Question Examples

A double-barreled question is a single question that contains two questions within it.

As a result, “respondents must answer two questions with one answer” Bradburn et al. (2004, p. 142).

The challenge of a double-barreled question is that it can be difficult to answer and may even be designed to cause someone confusion or cause them to have a slip of the tongue.

Double-barreled questions can occur in a survey, a questionnaire, a personality inventory, or an interview.

These types of measurement instruments are used in a very wide range of situations, from political opinion polls to customer satisfaction surveys, from market research to personality assessments and employment interviews.

Double-Barreled Questions Examples

• Course Evaluations: At the end of the academic term, students get a chance to evaluate the course and instructor. Hopefully this question will not appear: How would you rate this course and the professor’s effectiveness as a teacher?
• Rating a Restaurant: Sometimes restaurants have small cards on the table so customers can voice their opinion. Unfortunately, the following question might appear: I will come here again because the service is so good.
• Evaluating Software: On a scale from 0-10, how would you rate the quality and ease of use of our software? Obviously, this question is asking about two things: the quality of the software and how easy it is to use. Some software can be great, but figuring out how to use it can be a nightmare.
• In Job Satisfaction: Please indicate how much you agree or disagree with the following statement about your job:
  I love working here because everyone gets along so well. It is possible that everyone gets along, but a person might not like working at that company. It is also possible that a person loves working for that company, but not because people get along (maybe the pay is insanely great).
• Rating Student Progress: Teachers often rate their students’ progress by filling out a standardized form. One question might be: The student is motivated and shows interest in each lesson. The teacher then indicates to what extent the statement is reflective of the student’s behavior. The only problem is that a student can be motivated but only show interest in some types of lessons.
• During a Job Interview: The director of HR likes to ask the following question of applicants: Are you an employee that will be loyal to the company and work on the weekends?  Is it possible to be loyal but also have firm boundaries regarding not working on your days off?
• On a Date: Will you always be a gentleman and open the door for me? Answering that question can be risky. The problem is that being a gentleman can be expressed in a lot of ways, but they don’t necessarily include opening doors.
• Product Reviews: Some companies like to send a survey to new customers. One survey item might ask: Are you happy with your purchase and likely to recommend it to others? This is a double-barreled question because it is asking the customer if they are happy with their purchase, and also asking if they would recommend the product to others.  
• The Sincere Manager: Sometimes, a well-meaning manager will circulate through the restaurant and ask customers about their experience. It’s an informal survey of course, and includes questions such as: Are you enjoying your meal and service tonight? Fortunately, in this context, customers can easily dissect the question and provide an answer to each component.
• The Clever Lawyer: Sometimes, a lawyer tries to trip up a witness by asking confusing or challenging questions. For example: Can you honestly say that you did not know the item was stolen when you found it and tried to sell it to your neighbor? Of course, the other side’s lawyer can object on the grounds of this being a compound question.

Case Study: The European Social Survey

Although problematic issues regarding double-barreled questions have been well-known for decades, they can still be easily found, even in established surveys from respected institutions.

Menold (2020) calls our attention to an excellent example. The question below comes from the European Social Survey (ESS 2014, p. 3).

“And again, on an average weekday, how much of your time watching television is spent watching news or programs about politics and current affairs?”

This question contains two stimulus targets; one about politics and the other about current affairs. Although these topics sometimes overlap, they are distinct and fall into separate categories of programming.

Unfortunately, the instructions only allow for one response. The respondent much choose one response option among seven which range from “no time at all” to “more than 3 hours.”

So, what can a respondent do if they spend a little bit of time watching news about politics but a lot of time watching programs about current affairs?

Problems Caused by Double-Barreled Questions 

1. Issues Regarding Validity

The problem with double-barreled questions is that if a respondent indicates that they strongly agree with the question, the researcher has no way of knowing exactly which part of the question they agree with.

The more double-barreled questions, the less valid the measurement instrument.

This leads to two types of validity issues:

• Internal validity: First, the internal validity of the instrument is called into question. Internal validity refers to how well the individual questions actually measure what the scale purports to measure. Double-barreled questions lower internal validity.
• External validity: Secondly, the external validity of the instrument is weakened. External validity refers to how well the instrument applies to the population or state of affairs beyond the survey’s initial purpose.

If double-barreled questions are used in research, then the external validity of the study is also weakened.

There can be no external validity if there is no internal validity.

2. Issues Regarding Measurement Error

Very much related to internal validity is the issue of measurement error.

Ideally, when a respondent indicates the extent to which they agree or disagree with a statement on a survey by circling one of the response options, that circle is 100% reflective of their real opinion.

But, can a mark on a sheet of paper in the form of a circle really represent a person’s attitude about a given issue, especially one that is complex or controversial?

Unfortunately, this is a problem that exists in most social science research.

In the hard sciences, such as physics and chemistry, it is possible to measure constructs with incredible precision. Measuring the weight of an object can be so precise that it extends to dozens of decimal points.

In this case, there is almost zero measurement error. The readout on the scale represents the actual weight of the object.

This is an issue covered by classical test theory (CTT), which postulates that there is a true score (T) that could be attained if the measurement was 100% accurate (see De Champlain, 2010).

There is also an observed score (X), which is the number that is obtained when measuring something.

In survey questions, there is a great deal of error (E) in measurement, represented by the formula:

T = X + E

Asking a respondent to circle a number between 1 and 7 that represents their attitude or typical behavior is very imprecise.

There is a significant gap between T and X because there is a great deal of E.

The prevalence of double-barreled questions simply increases E. This issue is why psychologists spend a lot of time assessing the reliability and validity of paper-and-pencil measurement tools.

3. It Creates Cognitive Challenges

Encountering a double-barreled question creates momentary confusion in the respondent. Their cognitive processes are disrupted as they are unsure as to which of the two target stimuli presented in the question they should respond to.

As Tourangeau et al. (2002) outlined, responding to each question requires four steps: 1) comprehension, 2) retrieval, 3) judgment, and 4) response.

With a double-barreled question, step one requires the respondent to assess if the two target stimuli are similar or different in meaning. That takes significant cognitive resources.

Retrieval is also more complicated because the respondent must search through their long-term memory and locate information regarding two stimuli, not just one. The same challenge holds true for formulating a judgment and then generating a response.

These processes become more cognitively challenging because only one answer is allowed.

Each double-barreled question not only creates cognitive challenges in regard to that specific question, but it can have a carry-over effect on other questions, even ones that are SSQs.

4. Loss of Credibility

If there are a large number of double-barreled questions, respondents may begin to feel frustrated and/or lose respect for the entire process.

In fact, after a while, some respondents may fail to read each question carefully. Others may simply be more concerned about completing the survey as quickly as possible and escaping the entire situation.

When respondents take the process less seriously, it increases measurement error and undermines the internal and external validity of the results.

See More: Credibility Examples

Conclusion

A double-barreled question (double-barreled question) is a single question on a survey, questionnaire, or interview that asks about two aspects of one target stimulus. That target stimulus could be about a person’s opinion, their typical behavior, emotional experiences, or some aspect of their personality.

Double-barreled questions create challenges to the internal and external validity of the measurement instrument.

Each double-barreled question requires the respondent to exert excessive cognitive effort processing the question. This entails extra effort to comprehend the question, searching memory, formulating a final judgment, and then generating a response.

This can produce greater measurement errors. It can also make respondents feel frustrated and lose interest in the survey completely.

The simplest way to avoid these issues is to only incorporate single stimulus questions (SSQs).

References

Amsbary, J. H., & Powell, L. (2018). The Basics of Interviewing. In Interviewing in a Changing World (pp. 3-20). Routledge.

Bradburn, N. M., Sudman, S., & Wansink, B. (2004). Asking questions: The definitive guide to questionnaire design–for market research, political polls, and social and health questionnaires. John Wiley & Sons.

De Champlain, A. F. (2010). A primer on classical test theory and item response theory for assessments in medical education. Medical Education, 44(1), 109-117.

ESS, European Social Survey. (2014). ESS Round 7 Source Questionnaire. London: ESS ERIC Headquarters, Centre for Comparative Social Surveys, City University London. Available at: https://www.europeansocialsurvey.org/docs/round7/fieldwork/source/ESS7_source_main_questionnaire.pdf (accessed July, 2023).

Krosnick, J. A., & Presser, S. (2010). Question and questionnaire design. In Peter V. Marsden and James D. Wright Bingley (Eds.) Handbook of Survey Research (pp. 263-314). Emerald Group.

Menold, N. (2020). Double barreled questions: An analysis of the similarity of elements and effects on measurement quality. Journal of Official Statistics, 36(4), 855-886.

Tourangeau, R., L.J. Rips, and K.A. Rasinski. (2000). The psychology of survey response. Cambridge: Cambridge University Press.