Skip to content

Surveys · Questionnaire design

Double-Barrelled Questions and Why Respondents Cannot Answer Them

A double-barrelled question asks about two things at once but allows only one answer. This article explains why such questions produce uninterpretable data, how to spot the less obvious forms, and how to fix them.

ModQu research article · 1,555 words · 16 references (APA 7th edition)

Download PDF

"How satisfied are you with the speed and friendliness of our staff?" The question looks reasonable. It is short, it uses plain words and it addresses something a business genuinely wants to know. But a customer who found the staff friendly and slow has no good way to answer it. Neither does the researcher who later tries to interpret a score of three out of five.

This is a double-barrelled question: a single question that asks about two or more things but allows only one answer. The problem has been recognised in every major guide to questionnaire design for more than seventy years (Payne, 1951), and yet it remains one of the most common faults in practice. This article explains why double-barrelled questions damage data quality, how to recognise their less obvious forms and how to rewrite them.

Why respondents cannot answer them

Answering a survey question involves understanding it, retrieving relevant information, forming a judgement and mapping that judgement onto the answer options (Tourangeau et al., 2000). A double-barrelled question breaks the process at the judgement and mapping stages. A respondent who holds different views about the two parts must decide, privately and without guidance, how to turn two judgements into one answer.

Respondents resolve this in different ways. Some answer about whichever part seems more important to them. Some answer about the part mentioned first, or last. Some average their two views. Some choose the midpoint of the scale, or "don't know," because neither end fits. Some answer only if both parts apply, and others if either does (Bradburn et al., 2004). Because there is no way to tell which strategy each respondent used, the answers cannot be interpreted.

Reviews of the question-wording literature describe this as a failure of the basic rule that each question should measure a single, clearly defined thing (Schaeffer & Presser, 2003). The rule sounds obvious, but writers break it because they think about topics rather than about the separate judgements a respondent has to make. A questionnaire author who cares about "service" may naturally write about speed, friendliness and competence together, because to the author they are parts of one idea. To a respondent they are three separate experiences, and any one of them may have been good or bad.

Two-by-two grid for the question 'Are our staff fast and friendly?'. Respondents who think staff are fast and friendly answer yes clearly. Respondents who think staff are neither answer no clearly. Respondents who think staff are fast but unfriendly, or friendly but slow, have no correct answer and may say yes, no, choose the midpoint or skip.
Figure 1. Why a double-barrelled question fails. Only respondents whose views on the two parts agree can answer it unambiguously. Diagram by ModQu.

The damage is not only to the individual answer. Aggregated results become uninterpretable as well. If 60% of customers agree that staff are "fast and friendly," it is impossible to know whether a problem lies with speed, with friendliness or with neither. The score cannot guide action, which is usually the point of asking.

Respondents notice the difficulty. Bassili and Scott (1996) measured how long people took to answer different kinds of questions and found that double-barrelled questions took longer to answer than single-barrelled versions of the same content, a sign that respondents were working harder to resolve the ambiguity. Computational tools designed to flag problem questions treat such features as a source of comprehension difficulty (Graesser et al., 2006), and experiments have linked comprehension difficulty to longer response times and poorer answers (Lenzner et al., 2010).

Measurement quality

The effect on measurement quality has also been studied directly. Menold (2020) compared double-barrelled items with single-barrelled versions in experiments and found that double-barrelled items generally had lower measurement quality, particularly when the two parts were dissimilar. When the two elements are closely related, such as "clean and tidy," the damage is smaller, because most respondents hold similar views about both. When they are distinct, such as "affordable and reliable," the item effectively measures an unknown blend of two different constructs.

This matters for multi-item scales as well as single questions. A scale intended to measure one construct can be contaminated by an item that partly measures something else. Scale development guides therefore recommend reviewing each item for double-barrelled wording before data are collected (DeVellis, 2017).

Less obvious forms

The obvious form uses "and": "How satisfied are you with the price and quality?" Other forms are harder to spot.

Questions with lists of examples deserve the same scrutiny. "How often do you use social media such as Facebook, Instagram or TikTok?" may be intended as one question about social media in general, but some respondents will answer about the first service named, and others will add up their use of all three. Examples in a question shape how respondents understand it, so they should be chosen with care, or the question should name a single clear category (Krosnick & Presser, 2010).

Fowler (1995) suggests a simple test: if a respondent could reasonably answer "yes" to one part and "no" to another, the question needs to be split.

How to fix them

The standard remedy is to split the question into separate questions, each about one thing. "How satisfied are you with the speed of our staff?" and "How satisfied are you with the friendliness of our staff?" are both answerable and both actionable.

Splitting adds length, and questionnaire length is a cost. If the two parts are always expected to go together, and the researcher genuinely wants a combined judgement, an alternative is to ask about the overall concept directly, for example "How would you rate the service you received from our staff?", and accept that the answer is a summary judgement. What matters is that the question asks for one judgement, not two.

For hidden premises, the remedy is to separate the assumption from the opinion. First ask whether prices should be cut; then, if needed, ask what effects respondents expect. For reason clauses, ask about the behaviour first and the reason second: "Do you use our app?" followed by "What are the main reasons you use it?"

Converse and Presser (1986) emphasise that splitting is not just about clarity. It also produces richer data. Separate answers about speed and friendliness reveal which matters more for overall satisfaction, how the two relate, and where improvement is needed.

A short worked example shows the difference. Suppose a hotel asks guests one question: "Was your room clean and comfortable?" and 70% say yes. The manager cannot tell whether the remaining 30% had a dirty room, an uncomfortable bed or both. If the hotel instead asks two questions and finds that 92% rated cleanliness positively but only 72% rated comfort positively (illustrative numbers), the action is clear: invest in beds and pillows, not in housekeeping.

Self-administered questionnaires make the problem harder to catch. In an interview, a respondent who is puzzled by a question can say so, and a trained interviewer can record the difficulty. On paper or online there is no one to ask, so respondents simply choose an answer and move on, and the confusion leaves no trace in the data (Dillman et al., 2014). Questions that will be self-administered therefore need more careful review before fielding.

Finding them before fielding

Double-barrelled questions are easy to write and easy to overlook, especially for people who know the subject well. Several pretesting methods catch them (Presser et al., 2004):

Conclusion

A double-barrelled question asks respondents to give one answer to two questions. Those whose views differ on the two parts must invent their own rule for combining them, and the researcher cannot tell which rule they used. The result is data that cannot be interpreted and cannot guide action. The fix is usually simple: ask one thing at a time, separate premises from opinions, and test questions with real respondents before relying on them. Few improvements to a questionnaire are as cheap or as effective.

References

Bassili, J. N., & Scott, B. S. (1996). Response latency as a signal to question problems in survey research. Public Opinion Quarterly, 60(3), 390–399. https://doi.org/10.1086/297760

Bradburn, N. M., Sudman, S., & Wansink, B. (2004). Asking questions: The definitive guide to questionnaire design for market research, political polls, and social and health questionnaires (Rev. ed.). Jossey-Bass.

Converse, J. M., & Presser, S. (1986). Survey questions: Handcrafting the standardized questionnaire. SAGE. https://doi.org/10.4135/9781412986045

DeVellis, R. F. (2017). Scale development: Theory and applications (4th ed.). SAGE.

Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method (4th ed.). Wiley.

Fowler, F. J., Jr. (1995). Improving survey questions: Design and evaluation. SAGE.

Graesser, A. C., Cai, Z., Louwerse, M. M., & Daniel, F. (2006). Question Understanding Aid (QUAID): A web facility that tests question comprehensibility. Public Opinion Quarterly, 70(1), 3–22. https://doi.org/10.1093/poq/nfj012

Holbrook, A., Cho, Y. I., & Johnson, T. (2006). The impact of question and respondent characteristics on comprehension and mapping difficulties. Public Opinion Quarterly, 70(4), 565–595. https://doi.org/10.1093/poq/nfl027

Krosnick, J. A., & Presser, S. (2010). Question and questionnaire design. In P. V. Marsden & J. D. Wright (Eds.), Handbook of survey research (2nd ed., pp. 263–313). Emerald.

Lenzner, T., Kaczmirek, L., & Lenzner, A. (2010). Cognitive burden of survey questions and response times: A psycholinguistic experiment. Applied Cognitive Psychology, 24(7), 1003–1020. https://doi.org/10.1002/acp.1602

Menold, N. (2020). Double barreled questions: An analysis of the similarity of elements and effects on measurement quality. Journal of Official Statistics, 36(4), 855–886. https://doi.org/10.2478/jos-2020-0041

Payne, S. L. (1951). The art of asking questions. Princeton University Press.

Presser, S., Couper, M. P., Lessler, J. T., Martin, E., Martin, J., Rothgeb, J. M., & Singer, E. (2004). Methods for testing and evaluating survey questions. Public Opinion Quarterly, 68(1), 109–130. https://doi.org/10.1093/poq/nfh008

Schaeffer, N. C., & Presser, S. (2003). The science of asking questions. Annual Review of Sociology, 29, 65–88. https://doi.org/10.1146/annurev.soc.29.110702.110112

Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. Cambridge University Press. https://doi.org/10.1017/CBO9780511819322

Willis, G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. SAGE. https://doi.org/10.4135/9781412983655