Surveys · Questionnaire design
Double-Barrelled Questions and Why Respondents Cannot Answer Them
A double-barrelled question asks about two things at once but allows only one answer. This article explains why such questions produce uninterpretable data, how to spot the less obvious forms, and how to fix them.
"How satisfied are you with the speed and friendliness of our staff?" The question looks reasonable. It is short, it uses plain words and it addresses something a business genuinely wants to know. But a customer who found the staff friendly and slow has no good way to answer it. Neither does the researcher who later tries to interpret a score of three out of five.
This is a double-barrelled question: a single question that asks about two or more things but allows only one answer. The problem has been recognised in every major guide to questionnaire design for more than seventy years (Payne, 1951), and yet it remains one of the most common faults in practice. This article explains why double-barrelled questions damage data quality, how to recognise their less obvious forms and how to rewrite them.
Why respondents cannot answer them
Answering a survey question involves understanding it, retrieving relevant information, forming a judgement and mapping that judgement onto the answer options (Tourangeau et al., 2000). A double-barrelled question breaks the process at the judgement and mapping stages. A respondent who holds different views about the two parts must decide, privately and without guidance, how to turn two judgements into one answer.
Respondents resolve this in different ways. Some answer about whichever part seems more important to them. Some answer about the part mentioned first, or last. Some average their two views. Some choose the midpoint of the scale, or "don't know," because neither end fits. Some answer only if both parts apply, and others if either does (Bradburn et al., 2004). Because there is no way to tell which strategy each respondent used, the answers cannot be interpreted.
Reviews of the question-wording literature describe this as a failure of the basic rule that each question should measure a single, clearly defined thing (Schaeffer & Presser, 2003). The rule sounds obvious, but writers break it because they think about topics rather than about the separate judgements a respondent has to make. A questionnaire author who cares about "service" may naturally write about speed, friendliness and competence together, because to the author they are parts of one idea. To a respondent they are three separate experiences, and any one of them may have been good or bad.
The damage is not only to the individual answer. Aggregated results become uninterpretable as well. If 60% of customers agree that staff are "fast and friendly," it is impossible to know whether a problem lies with speed, with friendliness or with neither. The score cannot guide action, which is usually the point of asking.
Respondents notice the difficulty. Bassili and Scott (1996) measured how long people took to answer different kinds of questions and found that double-barrelled questions took longer to answer than single-barrelled versions of the same content, a sign that respondents were working harder to resolve the ambiguity. Computational tools designed to flag problem questions treat such features as a source of comprehension difficulty (Graesser et al., 2006), and experiments have linked comprehension difficulty to longer response times and poorer answers (Lenzner et al., 2010).
Measurement quality
The effect on measurement quality has also been studied directly. Menold (2020) compared double-barrelled items with single-barrelled versions in experiments and found that double-barrelled items generally had lower measurement quality, particularly when the two parts were dissimilar. When the two elements are closely related, such as "clean and tidy," the damage is smaller, because most respondents hold similar views about both. When they are distinct, such as "affordable and reliable," the item effectively measures an unknown blend of two different constructs.
This matters for multi-item scales as well as single questions. A scale intended to measure one construct can be contaminated by an item that partly measures something else. Scale development guides therefore recommend reviewing each item for double-barrelled wording before data are collected (DeVellis, 2017).
Less obvious forms
The obvious form uses "and": "How satisfied are you with the price and quality?" Other forms are harder to spot.
- "Or" questions. "Have you ever smoked cigarettes or cigars?" combines two behaviours. This may be acceptable if the researcher only needs to know about tobacco use generally, but not if the two need to be separated.
- Hidden premises. "Do you agree that the company should cut prices to attract new customers?" contains two claims: that prices should be cut, and that this would attract customers. A respondent who supports price cuts for a different reason, or doubts they would work, cannot answer cleanly.
- Reason clauses. "Do you use our app because it saves time?" asks both whether the respondent uses the app and why. Non-users and users with other reasons are both stuck.
- Compound behaviours. "How often do you read and reply to work email outside working hours?" mixes two activities that may occur at very different frequencies.
- Embedded evaluations. "How helpful was our excellent support team?" asks about helpfulness while asserting excellence, combining a double-barrelled problem with a leading one.
Questions with lists of examples deserve the same scrutiny. "How often do you use social media such as Facebook, Instagram or TikTok?" may be intended as one question about social media in general, but some respondents will answer about the first service named, and others will add up their use of all three. Examples in a question shape how respondents understand it, so they should be chosen with care, or the question should name a single clear category (Krosnick & Presser, 2010).
Fowler (1995) suggests a simple test: if a respondent could reasonably answer "yes" to one part and "no" to another, the question needs to be split.
How to fix them
The standard remedy is to split the question into separate questions, each about one thing. "How satisfied are you with the speed of our staff?" and "How satisfied are you with the friendliness of our staff?" are both answerable and both actionable.
Splitting adds length, and questionnaire length is a cost. If the two parts are always expected to go together, and the researcher genuinely wants a combined judgement, an alternative is to ask about the overall concept directly, for example "How would you rate the service you received from our staff?", and accept that the answer is a summary judgement. What matters is that the question asks for one judgement, not two.
For hidden premises, the remedy is to separate the assumption from the opinion. First ask whether prices should be cut; then, if needed, ask what effects respondents expect. For reason clauses, ask about the behaviour first and the reason second: "Do you use our app?" followed by "What are the main reasons you use it?"
Converse and Presser (1986) emphasise that splitting is not just about clarity. It also produces richer data. Separate answers about speed and friendliness reveal which matters more for overall satisfaction, how the two relate, and where improvement is needed.
A short worked example shows the difference. Suppose a hotel asks guests one question: "Was your room clean and comfortable?" and 70% say yes. The manager cannot tell whether the remaining 30% had a dirty room, an uncomfortable bed or both. If the hotel instead asks two questions and finds that 92% rated cleanliness positively but only 72% rated comfort positively (illustrative numbers), the action is clear: invest in beds and pillows, not in housekeeping.
Self-administered questionnaires make the problem harder to catch. In an interview, a respondent who is puzzled by a question can say so, and a trained interviewer can record the difficulty. On paper or online there is no one to ask, so respondents simply choose an answer and move on, and the confusion leaves no trace in the data (Dillman et al., 2014). Questions that will be self-administered therefore need more careful review before fielding.
Finding them before fielding
Double-barrelled questions are easy to write and easy to overlook, especially for people who know the subject well. Several pretesting methods catch them (Presser et al., 2004):
- Systematic review. Read each question looking specifically for "and," "or," "because," "to" and other connecting words, and ask whether each connects two separable ideas.
- Cognitive interviews. Ask a few members of the target population to explain how they arrived at their answer. People asked about a double-barrelled question often say "it depends" or explain that they were thinking about only one part (Willis, 2005).
- Respondent difficulty measures. Studies of comprehension problems show that some question features consistently cause difficulty across respondent groups, which makes it worth checking for them systematically rather than waiting for complaints (Holbrook et al., 2006).
- Pilot data. Unusually high rates of midpoint or "don't know" answers on an item can signal that respondents found it impossible to answer.
Conclusion
A double-barrelled question asks respondents to give one answer to two questions. Those whose views differ on the two parts must invent their own rule for combining them, and the researcher cannot tell which rule they used. The result is data that cannot be interpreted and cannot guide action. The fix is usually simple: ask one thing at a time, separate premises from opinions, and test questions with real respondents before relying on them. Few improvements to a questionnaire are as cheap or as effective.
References
Bassili, J. N., & Scott, B. S. (1996). Response latency as a signal to question problems in survey research. Public Opinion Quarterly, 60(3), 390–399. https://doi.org/10.1086/297760
Bradburn, N. M., Sudman, S., & Wansink, B. (2004). Asking questions: The definitive guide to questionnaire design for market research, political polls, and social and health questionnaires (Rev. ed.). Jossey-Bass.
Converse, J. M., & Presser, S. (1986). Survey questions: Handcrafting the standardized questionnaire. SAGE. https://doi.org/10.4135/9781412986045
DeVellis, R. F. (2017). Scale development: Theory and applications (4th ed.). SAGE.
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method (4th ed.). Wiley.
Fowler, F. J., Jr. (1995). Improving survey questions: Design and evaluation. SAGE.
Graesser, A. C., Cai, Z., Louwerse, M. M., & Daniel, F. (2006). Question Understanding Aid (QUAID): A web facility that tests question comprehensibility. Public Opinion Quarterly, 70(1), 3–22. https://doi.org/10.1093/poq/nfj012
Holbrook, A., Cho, Y. I., & Johnson, T. (2006). The impact of question and respondent characteristics on comprehension and mapping difficulties. Public Opinion Quarterly, 70(4), 565–595. https://doi.org/10.1093/poq/nfl027
Krosnick, J. A., & Presser, S. (2010). Question and questionnaire design. In P. V. Marsden & J. D. Wright (Eds.), Handbook of survey research (2nd ed., pp. 263–313). Emerald.
Lenzner, T., Kaczmirek, L., & Lenzner, A. (2010). Cognitive burden of survey questions and response times: A psycholinguistic experiment. Applied Cognitive Psychology, 24(7), 1003–1020. https://doi.org/10.1002/acp.1602
Menold, N. (2020). Double barreled questions: An analysis of the similarity of elements and effects on measurement quality. Journal of Official Statistics, 36(4), 855–886. https://doi.org/10.2478/jos-2020-0041
Payne, S. L. (1951). The art of asking questions. Princeton University Press.
Presser, S., Couper, M. P., Lessler, J. T., Martin, E., Martin, J., Rothgeb, J. M., & Singer, E. (2004). Methods for testing and evaluating survey questions. Public Opinion Quarterly, 68(1), 109–130. https://doi.org/10.1093/poq/nfh008
Schaeffer, N. C., & Presser, S. (2003). The science of asking questions. Annual Review of Sociology, 29, 65–88. https://doi.org/10.1146/annurev.soc.29.110702.110112
Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. Cambridge University Press. https://doi.org/10.1017/CBO9780511819322
Willis, G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. SAGE. https://doi.org/10.4135/9781412983655