Inquiry · The art of the question
Asking Better Questions: A Practical Framework
Good questions can be learned. This article sets out a six-step framework for asking better questions in research, management and everyday inquiry, drawing on evidence from survey methodology, cognitive psychology and the study of problem finding.
Most people learn to ask questions the way they learn to walk: early, without instruction and without thinking much about it afterwards. Yet the questions people ask determine what they find out. A poorly framed question can send a research project in the wrong direction, produce answers that cannot be used or confirm a mistaken belief. A well-framed question can open up a problem, draw out information that would otherwise stay hidden and lead to better decisions.
The good news is that asking good questions is a skill that can be learned and improved. Survey methodologists have spent decades studying how the wording and structure of questions affect the answers. Cognitive psychologists have studied how people understand and answer questions, and how they test their own ideas. Researchers on creativity and problem solving have studied how people find and frame problems in the first place. This article draws these strands together into a practical framework of six steps, summarised in Figure 1, that can be applied to a research study, a management decision or a single conversation.
Step 1: Clarify the purpose
The first step comes before any question is written: decide what the answer is for. A question that is interesting is not necessarily useful. The test of a useful question is that its answer would change what someone does, believes or decides.
Survey methodologists make the same point. Bradburn et al. (2004) advise that every question in a questionnaire should be justified by the research objective it serves, and that questions included "because it would be interesting to know" should be cut. Each extra question adds burden for the respondent and dilutes attention to the questions that matter. A simple discipline is to write down, before drafting any question, the decision the research will inform and what the team would do differently depending on the answer. If no answer would change anything, the question is not worth asking.
Clarifying purpose also means distinguishing between kinds of knowledge. Some questions seek a description (how many customers use the feature?), some an explanation (why do they stop using it?), some a prediction (will they use a new version?) and some an evaluation (was the change worth making?). Each calls for a different design. A common failure is to ask a descriptive question when the decision really needs an explanation, and then to read causes into a description.
Step 2: Surface the assumptions
Every question rests on assumptions. Philosophers call them presuppositions: things a question takes for granted. "Why did the campaign fail?" presupposes that it failed. "Which of these three suppliers should we choose?" presupposes that one of the three is the right answer. R. G. Collingwood (1939) argued that every question has presuppositions, and that much confused thinking comes from failing to notice them. Jaakko Hintikka (2007) built a theory of inquiry around this observation: in his model, a question can only be answered if its presupposition is true, so the inquirer must first establish that it is.
Hidden presuppositions can also shape the answers people give. Elizabeth Loftus and John Palmer (1974) showed that asking witnesses how fast cars were going when they "smashed into" each other, rather than when they "hit" each other, raised speed estimates and led more witnesses to remember broken glass that was never there. The verb carried an assumption about the severity of the accident. In management, Thomas Wedell-Wedellsborg (2017) found that many organisations solve the wrong problems because they accept the first framing of a problem without examining what it assumes. His remedy is to ask, before solving a problem, "Is this the right problem to solve?" and "What else could be going on?"
A practical technique is to write a question down and then list everything it takes for granted. Each assumption can then be checked, or the question rewritten to remove it. "Why did the campaign fail?" might become "How did the campaign perform against its objectives, and what explains the result?"
Step 3: Open before narrowing
Once the purpose and assumptions are clear, the next step is to decide how wide the question should be. Closed questions, which offer a fixed set of answers, are efficient and comparable, but they can only find what the questioner thought to offer. Howard Schuman and Stanley Presser (1979) showed in national survey experiments that the answers to open and closed versions of the same question differ substantially, and that answers not offered on a closed list largely disappear. Open questions reveal possibilities the researcher had not imagined.
The research on problem finding points the same way. Jacob Getzels and Mihaly Csikszentmihalyi (1976) studied art students and found that those who spent longer exploring materials and possibilities before settling on what to draw produced work that experts judged more original, and that this "problem-finding" behaviour predicted their success as artists years later. In a different domain, the same principle applies: time spent exploring the question before committing to an answer pays off.
The practical rule is to start broad. Early in an inquiry, ask open questions that let the subject matter, the data or the people being studied show what is important. Only then narrow to specific, closed questions that measure what the open phase has revealed. A research programme might begin with exploratory interviews and move to a structured survey; a single interview might begin with "Tell me about…" and move to "Was it more or less than last year?" This sequence is sometimes called the funnel.
Step 4: Avoid confirmation and choose the form
A particular danger in framing questions is that people tend to ask questions that can only confirm what they already believe. Peter Wason (1960) asked people to discover a rule governing sequences of three numbers, given the example 2, 4, 6. Most people formed a hypothesis, such as "increasing by two", and tested only sequences that fitted it, such as 8, 10, 12. Because the actual rule was simply "any increasing sequence", these tests always produced a yes, and people announced the wrong rule with confidence. Raymond Nickerson (1998) reviewed the extensive research on this confirmation bias and found it in many domains, from scientific reasoning to medical diagnosis.
Joshua Klayman and Young-Won Ha (1987) refined the analysis. They showed that seeking confirming cases is not always irrational, but that it fails when the true answer is broader than the hypothesis. The remedy is to ask questions whose answers could prove the hypothesis wrong. Instead of asking customers "Would you use a feature that did X?", which invites agreement, a researcher might ask "How do you currently handle X?", which reveals whether there is a problem at all.
Once the scope is clear, the form of each question follows. Closed questions suit measurement; open questions suit discovery; probing follow-ups ("Can you give me an example?") deepen an answer; and reflective questions ("What would you do differently?") invite people to examine their own reasoning. Kvale and Brinkmann (2009) recommend that interview questions be short, simple and concrete, asking about specific experiences rather than general opinions, because concrete accounts are more reliable.
Wording matters as much as form. Norbert Schwarz (1999) showed that respondents use every feature of a question, including the response options, as information about what is being asked. Amos Tversky and Daniel Kahneman (1981) showed that describing the same choice in terms of gains or losses reverses people's preferences. Some practical wording rules follow from this research:
- Use simple, familiar words, and define any term that could be read in more than one way.
- Ask about one thing at a time. "How satisfied are you with the price and quality?" is two questions.
- Avoid leading language that suggests a preferred answer.
- Make response options balanced, covering the full range of possible answers.
- Specify the time period ("in the past 30 days") rather than leaving it vague.
Step 5: Check that the respondent can answer
A question can be well framed and well worded and still fail if the person answering cannot answer it accurately. Roger Tourangeau and colleagues (2000) described answering a question as four tasks: understanding it, retrieving relevant information from memory, forming a judgement and reporting it in the format offered. A failure at any stage produces a poor answer.
Comprehension fails when a question uses unfamiliar terms or can be read in more than one way. Retrieval fails when the question asks about events that are too distant, too frequent or too unremarkable to remember. Seymour Sudman and Norman Bradburn (1973) showed that memory errors grow with the length of the recall period, which is why good surveys ask about recent, specific periods. Judgement fails when people are asked to explain causes they cannot observe, such as the reasons for their own preferences. Reporting fails when the answer options do not fit what the person wants to say, or when social pressure makes some answers uncomfortable.
Jon Krosnick (1991) added that respondents do not always try their hardest. When a question is difficult, or when they are tired or unmotivated, many satisfice: they give an answer that seems acceptable without working through all four tasks. They agree with statements, pick the first reasonable option or say "don't know". Short, concrete questions placed early in an interview or survey reduce satisficing.
Sensitive topics add a further difficulty. Questions about income, health, illegal behaviour or socially disapproved opinions invite people to edit their answers. Roger Tourangeau and Ting Yan (2007) reviewed the evidence and found that misreporting on sensitive topics is common and largely deliberate: people under-report behaviours they expect to be judged for and over-report those that are approved of, such as voting. They also found that the way a question is asked makes a difference. Self-completed questionnaires, especially on a computer, produce more candid answers than questions asked aloud by an interviewer, and assurances of confidentiality help when they are credible. Wording can help too. A question that normalises the behaviour ("Many people find it hard to keep up with every bill. In the past year, have you missed any payments?") makes an honest answer easier than a blunt one.
The same principles apply in organisations. An employee asked by their manager, in a meeting, whether a project is on track is answering a sensitive question, and the answer may reflect the pressure of the setting more than the state of the project. Anonymous channels, a track record of responding well to bad news and questions that make it easy to report problems ("What is the biggest risk to this deadline?") all produce more accurate information than "Is everything on schedule?"
The practical test is to imagine a specific respondent answering the question. Would they understand every word in the same way the researcher does? Would they remember the information? Can they actually know the answer, or would they be guessing? Is there an easy way out that they might take?
Step 6: Test and follow up
No question is finished until it has been tried. Survey methodologists treat pre-testing as essential. Gordon Willis (2005) describes cognitive interviewing, in which a small number of people answer draft questions while thinking aloud or responding to probes such as "What does the word 'regularly' mean to you here?" These interviews regularly reveal misunderstandings that the question writers never anticipated. Presser et al. (2004) reviewed the methods available for testing survey questions, from expert review to field experiments, and concluded that different methods catch different problems, so that more than one should be used for important questions.
In conversations and interviews, testing happens in real time through follow-up. The first answer to a question is rarely the full answer. Probing questions such as "What makes you say that?" and "Can you give me an example?" turn a superficial response into a useful one. Huang et al. (2017) found that people who ask more follow-up questions are also better liked, because follow-ups show that they are listening. Follow-up also reveals when the original question was misunderstood, so that it can be asked again in a different way.
Finally, follow-up includes reflecting on what the questions produced. After a study or a meeting, it is worth asking which questions yielded the most useful answers, which fell flat and what the team wishes it had asked. These lessons improve the next round of questions.
An example: applying the framework
A hypothetical software company is concerned that customers are cancelling their subscriptions. The first draft of its research question is: "Why are customers leaving because of our prices?"
- Purpose. The team asks what decision the research will inform. The answer is whether to change the pricing structure, improve the product or change how new customers are brought on board. The question needs to cover all three possibilities.
- Assumptions. The draft assumes that price is the reason. The team rewrites it to remove that assumption: "Why do customers cancel, and which reasons are most common among customers we most want to keep?"
- Open before narrowing. The team starts with twenty open interviews with recent cancellers, asking them to describe the period before they cancelled. Price comes up, but so do a confusing setup process and the loss of a key internal champion at the customer's company.
- Avoid confirmation and choose the form. The team designs a survey that lists all the reasons found in the interviews, in random order, with an "other" option, so that price has no special prominence. It asks about specific events ("In the month before you cancelled, did any of the following happen?") rather than general opinions.
- Respondent. The team limits the survey to customers who cancelled in the past three months, so that memories are fresh, and avoids asking people to rate the importance of reasons they may not be able to introspect on.
- Test and follow up. Five cognitive interviews reveal that "onboarding" is not a word customers use, so it is replaced with "getting set up". After the survey, the team reviews which questions discriminated between customers who might return and those who would not.
The final research identifies setup problems as the main driver of early cancellations, and price as a significant factor only for a small group of customers. The company's response is different, and cheaper, than a price cut would have been.
Habits of good questioners
Beyond the six steps, research points to some general habits. Warren Berger (2014) observed that innovators tend to cycle through three kinds of questions: "Why?" (understanding a situation), "What if?" (imagining possibilities) and "How?" (putting an idea into action). Rothstein and Santana (2011) showed that people of all ages can learn to generate, improve and prioritise their own questions through a structured technique, and that doing so increases their engagement and ownership of inquiry.
Arthur Graesser and Natalie Person (1994) found that most questions people ask are shallow requests for facts, and that deeper questions about causes, consequences and comparisons are rarer but more strongly linked to understanding. A useful self-check is to look at a list of questions and count how many ask "why", "how" or "what if".
Finally, good questioning depends on an environment in which questions are welcome. Amy Edmondson (1999) found that teams learn more when members feel safe to ask questions, admit uncertainty and raise concerns. In such environments, the person who asks the obvious question that no one else will ask is valued rather than ignored. Leaders who model good questioning, by asking genuine questions and responding well to the answers, make it easier for others to do the same.
Conclusion
Asking better questions is not a matter of talent or inspiration. It is a skill with identifiable components: knowing what the answer is for, examining what the question assumes, opening the inquiry before narrowing it, guarding against confirmation, wording questions so that people can understand and answer them, and testing and following up. Each of these steps is supported by research, and each can be practised. Organisations that invest in the quality of their questions find that the quality of their answers, and their decisions, improves with it.
References
Berger, W. (2014). A more beautiful question: The power of inquiry to spark breakthrough ideas. Bloomsbury.
Bradburn, N. M., Sudman, S., & Wansink, B. (2004). Asking questions: The definitive guide to questionnaire design (Rev. ed.). Jossey-Bass.
Collingwood, R. G. (1939). An autobiography. Oxford University Press.
Edmondson, A. (1999). Psychological safety and learning behavior in work teams. Administrative Science Quarterly, 44(2), 350–383. https://doi.org/10.2307/2666999
Getzels, J. W., & Csikszentmihalyi, M. (1976). The creative vision: A longitudinal study of problem finding in art. Wiley.
Graesser, A. C., & Person, N. K. (1994). Question asking during tutoring. American Educational Research Journal, 31(1), 104–137. https://doi.org/10.3102/00028312031001104
Hintikka, J. (2007). Socratic epistemology: Explorations of knowledge-seeking by questioning. Cambridge University Press. https://doi.org/10.1017/CBO9780511619298
Huang, K., Yeomans, M., Brooks, A. W., Minson, J., & Gino, F. (2017). It doesn't hurt to ask: Question-asking increases liking. Journal of Personality and Social Psychology, 113(3), 430–452. https://doi.org/10.1037/pspi0000097
Klayman, J., & Ha, Y.-W. (1987). Confirmation, disconfirmation, and information in hypothesis testing. Psychological Review, 94(2), 211–228. https://doi.org/10.1037/0033-295X.94.2.211
Krosnick, J. A. (1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology, 5(3), 213–236. https://doi.org/10.1002/acp.2350050305
Kvale, S., & Brinkmann, S. (2009). InterViews: Learning the craft of qualitative research interviewing (2nd ed.). Sage.
Loftus, E. F., & Palmer, J. C. (1974). Reconstruction of automobile destruction: An example of the interaction between language and memory. Journal of Verbal Learning and Verbal Behavior, 13(5), 585–589. https://doi.org/10.1016/S0022-5371(74)80011-3
Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2), 175–220. https://doi.org/10.1037/1089-2680.2.2.175
Presser, S., Couper, M. P., Lessler, J. T., Martin, E., Martin, J., Rothgeb, J. M., & Singer, E. (2004). Methods for testing and evaluating survey questions. Public Opinion Quarterly, 68(1), 109–130. https://doi.org/10.1093/poq/nfh008
Rothstein, D., & Santana, L. (2011). Make just one change: Teach students to ask their own questions. Harvard Education Press.
Schuman, H., & Presser, S. (1979). The open and closed question. American Sociological Review, 44(5), 692–712. https://doi.org/10.2307/2094521
Schwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54(2), 93–105. https://doi.org/10.1037/0003-066X.54.2.93
Sudman, S., & Bradburn, N. M. (1973). Effects of time and memory factors on response in surveys. Journal of the American Statistical Association, 68(344), 805–815. https://doi.org/10.1080/01621459.1973.10481428
Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. Cambridge University Press. https://doi.org/10.1017/CBO9780511819322
Tourangeau, R., & Yan, T. (2007). Sensitive questions in surveys. Psychological Bulletin, 133(5), 859–883. https://doi.org/10.1037/0033-2909.133.5.859
Tversky, A., & Kahneman, D. (1981). The framing of decisions and the psychology of choice. Science, 211(4481), 453–458. https://doi.org/10.1126/science.7455683
Wason, P. C. (1960). On the failure to eliminate hypotheses in a conceptual task. Quarterly Journal of Experimental Psychology, 12(3), 129–140. https://doi.org/10.1080/17470216008416717
Wedell-Wedellsborg, T. (2017). Are you solving the right problems? Harvard Business Review, 95(1), 76–83.
Willis, G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. Sage. https://doi.org/10.4135/9781412983655