Intelligence · Analysis and tradecraft
The Role of Imagination and Creativity in Intelligence Hypothesis Generation
An analyst cannot test an explanation that no one has thought of. This article explains why generating a wide range of hypotheses is critical in intelligence work, what research on creativity shows, which group methods help and which hinder, and how to build imagination into analytic routines.
Intelligence analysis is often described as testing hypotheses against evidence. But that description skips the first and perhaps most important step: thinking of the hypotheses in the first place. If the true explanation is not among those considered, no amount of careful testing will find it. Heuer (1999) argued that a failure to generate the right hypothesis is one of the most serious sources of analytic error, and that analysts typically consider too few alternatives. This is where imagination and creativity enter intelligence work.
A failure of imagination
The 9/11 Commission concluded that the most important failure before the attacks was one of imagination: American agencies had not seriously considered that terrorists might hijack aircraft and fly them into buildings, although the idea had appeared in some earlier reporting (National Commission on Terrorist Attacks Upon the United States, 2004). The commission recommended finding ways of "routinizing, even bureaucratizing, the exercise of imagination". The phrase is paradoxical, but the point is serious. Imagination cannot be left to chance; it needs methods and time.
What creativity research shows
Guilford (1950), in a landmark address to the American Psychological Association, distinguished convergent thinking, which narrows towards a single correct answer, from divergent thinking, which generates many possible answers. Much analytic training emphasises convergent thinking: weighing evidence and reaching a conclusion. Hypothesis generation depends on divergent thinking.
Mednick (1962) proposed that creativity arises from forming new combinations of remote associations, ideas that are not usually linked. People who can draw connections between distant ideas are more likely to produce original solutions. For analysts, this suggests the value of broad knowledge and of exposure to different fields, cases and perspectives.
Amabile (1996) argued that creativity depends on three components: expertise in the relevant domain, creative thinking skills and intrinsic motivation. Amabile also showed that the work environment matters. Tight deadlines, harsh evaluation and close supervision tend to reduce creativity, while autonomy and encouragement support it. Intelligence organisations under constant time pressure may therefore be suppressing exactly the thinking they need.
Brainstorming and its limits
Osborn (1953) introduced brainstorming with a set of rules: defer judgement, aim for quantity, welcome unusual ideas and build on others' suggestions. The method became hugely popular. But research soon showed a surprising result. Diehl and Stroebe (1987) found that people working alone and then pooling their ideas, so-called nominal groups, produced more ideas than the same number of people brainstorming together. The main cause was production blocking: in a group, only one person can speak at a time, and others forget or suppress their ideas while waiting.
Delbecq et al. (1975) developed the nominal group technique to address this problem. Members first write ideas silently and independently, then share them in turn, and only afterwards discuss and evaluate them. The technique keeps the benefits of pooling while avoiding blocking and the influence of dominant members.
Nemeth et al. (2001) found that genuine disagreement in a group stimulated more ideas than either consensus or a role-played devil's advocate. Diversity of views, not just of people, fuels imagination.
Methods for analysts
Pherson and Heuer (2021) describe several structured techniques for generating hypotheses and imagining alternatives:
- Structured brainstorming: a version of the nominal group technique in which participants write ideas on cards before discussion.
- Outside-in thinking: starting from broad forces, such as technology, economics and politics, and asking how they might affect the issue, rather than starting from the issue itself.
- "What if?" analysis: assuming an unexpected event has happened and working backwards to explain how it could have come about.
- The premortem: imagining that a judgement has turned out to be wrong and explaining why (Klein, 2007).
Feduzi and Runde (2014) suggest a deliberate search for alternative hypotheses as a way of uncovering "unknown unknowns": each new hypothesis implies new questions, and the questions reveal what has not yet been considered.
Common obstacles
Several features of analytic work discourage imagination. Time pressure pushes analysts to adopt the first plausible explanation. Experts may dismiss unusual ideas because they conflict with long experience, which is often right but occasionally blinds them to genuine change. Hierarchies discourage junior staff from offering odd suggestions in front of senior colleagues. And because unlikely hypotheses usually turn out to be wrong, analysts who raise them can feel they are wasting time. Amabile (1996) found that people are more creative when they feel their ideas will be valued rather than judged harshly. Leaders can help by asking explicitly for unconventional explanations and by treating them as a normal part of good analysis, not as a distraction.
An illustration
A Canadian bank notices that customers in one region are closing savings accounts at twice the usual rate. The first explanation offered is that a rival has launched a better interest rate. Before testing it, the analysis team spends twenty minutes generating alternatives independently. The pooled list includes the rival's offer, a local employer laying off staff, a change in the bank's own fees, a technical problem with the mobile app in that region, a social media rumour about the bank's stability and the closure of a branch. Testing each against the evidence shows that the rival's rate change happened after the closures began. The pattern matches the branch closure and the app problem, which affected the same customers. Without the deliberate search for alternatives, the bank might have matched the rival's rate at considerable cost and left the real problems unsolved.
Building imagination into routines
Practical steps include setting aside time at the start of each major assessment for hypothesis generation, requiring at least three alternatives in every analysis, inviting people from outside the team, and recording rejected hypotheses so that they can be revisited if evidence changes. Organisations should also review past surprises and ask whether the eventual explanation had been considered at all. Where it had been considered and rejected, the review should ask why; where it had never been considered, the review should ask what kind of source, person or method might have suggested it. Over time, these reviews show an organisation where its imagination tends to fail.
Conclusion
Imagination is not the opposite of rigour in intelligence; it is its precondition. Research on creativity shows that divergent thinking can be encouraged by independent idea generation, diverse perspectives, genuine dissent and an environment that allows time to think. Structured techniques make imagination routine rather than accidental, and they ensure that the explanation that later proves correct was at least on the list.
References
Amabile, T. M. (1996). Creativity in context. Westview Press.
Delbecq, A. L., Van de Ven, A. H., & Gustafson, D. H. (1975). Group techniques for program planning: A guide to nominal group and Delphi processes. Scott, Foresman.
Diehl, M., & Stroebe, W. (1987). Productivity loss in brainstorming groups: Toward the solution of a riddle. Journal of Personality and Social Psychology, 53(3), 497–509. https://doi.org/10.1037/0022-3514.53.3.497
Feduzi, A., & Runde, J. (2014). Uncovering unknown unknowns: Towards a Baconian approach to management decision-making. Organizational Behavior and Human Decision Processes, 124(2), 268–283. https://doi.org/10.1016/j.obhdp.2014.04.001
Guilford, J. P. (1950). Creativity. American Psychologist, 5(9), 444–454. https://doi.org/10.1037/h0063487
Heuer, R. J., Jr. (1999). Psychology of intelligence analysis. Center for the Study of Intelligence, Central Intelligence Agency.
Klein, G. (2007). Performing a project premortem. Harvard Business Review, 85(9), 18–19.
Mednick, S. A. (1962). The associative basis of the creative process. Psychological Review, 69(3), 220–232. https://doi.org/10.1037/h0048850
National Commission on Terrorist Attacks Upon the United States. (2004). The 9/11 Commission report. W. W. Norton.
Nemeth, C., Brown, K., & Rogers, J. (2001). Devil's advocate versus authentic dissent: Stimulating quantity and quality. European Journal of Social Psychology, 31(6), 707–720. https://doi.org/10.1002/ejsp.58
Osborn, A. F. (1953). Applied imagination: Principles and procedures of creative thinking. Scribner.
Pherson, R. H., & Heuer, R. J., Jr. (2021). Structured analytic techniques for intelligence analysis (3rd ed.). CQ Press.