Skip to content

Intelligence · Analysis and tradecraft

"Intelligence Failures: Case Studies and Lessons"

Studies of major intelligence failures, from Pearl Harbor to Iraq, show recurring patterns in how organisations misread the evidence before them. This article reviews five well-documented cases, the causes identified by later inquiries and the lessons for any organisation that relies on intelligence.

ModQu research article · 1,471 words · 15 references (APA 7th edition)

Download PDF

Intelligence failures are rare in the sense that most daily judgements turn out reasonably well. But when they occur, their consequences can be enormous, and they are studied intensively afterwards. Official inquiries and scholarly research on these failures form one of the richest bodies of evidence about how organisations interpret information under uncertainty. This article reviews five well-documented cases, identifies the patterns that recur across them and draws lessons that apply to businesses as well as governments.

What counts as a failure

An intelligence failure is usually defined as a misunderstanding of a situation that leads to actions harmful to the organisation's interests. The error may lie in collection, analysis, communication or the response of decision-makers, and most failures involve several of these. Betts (1978) argued that some failures are inevitable. Evidence is ambiguous, adversaries deceive, and every remedy for one kind of error creates the risk of another: more warnings lead to more false alarms, which lead users to ignore warnings. The goal, therefore, is not to eliminate failure but to make it less frequent and less severe.

Table of five intelligence failures: Pearl Harbor 1941, the Cuban missiles 1962, the Yom Kippur War 1973, the 9/11 attacks 2001 and Iraqi weapons of mass destruction 2002, each with the mistaken belief and the main cause identified later.
Figure 1. Five well-studied intelligence failures. In each case, relevant information existed; the failure lay mainly in how it was interpreted, shared or acted upon. Compiled from Wohlstetter (1962), Knorr (1964), Bar-Joseph (2005), the 9/11 Commission (2004) and the WMD Commission (2005); table by ModQu.

Pearl Harbor, 1941

Wohlstetter (1962) conducted the classic study of the Japanese attack on the American fleet at Pearl Harbor. Wohlstetter found that American agencies had, between them, a great deal of information pointing towards an attack, including decrypted Japanese diplomatic messages. But the relevant signals were scattered among many irrelevant or misleading items, which Wohlstetter called noise. Responsibility was divided between the army, the navy and Washington, and each assumed the others were alert. Most American officials expected a Japanese move in South-East Asia, and evidence was interpreted to fit that expectation.

The Cuban missiles, 1962

In September 1962, an American national intelligence estimate judged that the Soviet Union was unlikely to place offensive missiles in Cuba. Weeks later, reconnaissance photographs showed that it was doing exactly that. Knorr (1964) concluded that analysts had judged the deployment unlikely because it would have been a risky and irrational move from the American point of view. They did not sufficiently consider how the situation looked from Moscow. Allison and Zelikow (1999) later showed how different models of government decision-making lead to different explanations of the Soviet choice, underlining the danger of assuming that an adversary reasons as one does oneself.

The Yom Kippur War, 1973

Israeli military intelligence had detailed evidence of Egyptian and Syrian preparations before the attack of October 1973, including warnings from a highly placed source. Bar-Joseph (2005) shows that analysts relied on a fixed belief, known as "the concept", that Egypt would not go to war until it could defeat Israel's air force. Evidence of preparations was interpreted as exercises or bluff. The head of military intelligence was confident, persuasive and reluctant to revise a settled view, and the system had few mechanisms for challenge. Kam (1988) describes how this pattern, in which strong beliefs override accumulating warnings, recurs in many surprise attacks.

The 9/11 attacks, 2001

The 9/11 Commission found that American agencies had collected warnings about al-Qaeda's intentions and about individual hijackers but had not connected them (National Commission on Terrorist Attacks Upon the United States, 2004). Information was held in separate agencies that did not share it effectively. The commission also described a "failure of imagination": few analysts had seriously considered that hijacked aircraft might be used as weapons on a large scale. Zegart (2007) argued that the deeper cause lay in organisational structures and incentives in the CIA and FBI, which had failed to adapt to the threat from terrorism despite repeated warnings and reform proposals.

Iraqi weapons of mass destruction, 2002

Before the 2003 invasion of Iraq, American and British intelligence assessed that Iraq possessed chemical and biological weapons and was reconstituting its nuclear programme. After the invasion, no such weapons were found. The American commission concluded that analysts had relied on the assumption that Iraq retained weapons because it had had them before and was behaving deceptively, and that key sources were unreliable but their reports were given too much weight (Commission on the Intelligence Capabilities of the United States Regarding Weapons of Mass Destruction, 2005). The British review found that caveats in the underlying intelligence were lost as judgements were summarised for public use (Butler, 2004).

Jervis (2010) argued that the conclusion itself was reasonable given the evidence, but that the analysis was poorly done: analysts did not ask what evidence would distinguish between the competing explanations, nor how Iraq's behaviour might look if it had no weapons but wished to appear as though it did. Pillar (2011), who was a senior American analyst at the time, describes how the policy climate shaped which questions were asked and how the intelligence was used.

Patterns across the cases

Although the circumstances differ, the same causes appear repeatedly.

  1. Fixed beliefs. In almost every case, a strong prior view shaped the interpretation of new evidence. Heuer (1999) explains why: mental models form quickly and change slowly, and evidence that does not fit is easily discounted.
  2. Mirror imaging. Analysts assumed that adversaries would reason as they did, as in Cuba and in 1973.
  3. Information not connected. Relevant facts were held by different people or agencies, as at Pearl Harbor and before 9/11.
  4. Weak challenge. Few mechanisms existed to test the prevailing view, and dissenting voices were weak or absent.
  5. Lost caveats. Uncertainty in the underlying evidence was not carried through to the final judgement, as in the Iraq case.
  6. Organisational inertia. Structures and incentives that made sense for earlier problems were slow to adapt (Zegart, 2007).

Vaughan (1996), studying the loss of the space shuttle Challenger, described a related process in a non-intelligence setting: the normalisation of deviance, in which small warning signs are repeatedly accepted until they come to seem normal. The concept applies equally to intelligence organisations that grow used to ambiguous signals.

The problem of hindsight

Studying failures carries its own risk. Once the outcome is known, the signals that pointed to it seem obvious, and the noise that surrounded them is forgotten. Wohlstetter (1962) stressed that it is much easier after the event to sort relevant from irrelevant signals than it was beforehand. Betts (1978) added that inquiries rarely examine the many occasions on which similar warnings were correctly dismissed as false alarms. A fair assessment asks not only what was missed but what a reasonable analyst could have concluded at the time, with the information and the competing demands then present. Lessons drawn without this discipline tend to produce reforms that would have prevented the last failure but not the next.

A business illustration

A Canadian electronics retailer saw online competitors growing for several years but judged that customers would keep buying expensive items in stores, where they could see and test them. Store managers reported that shoppers were examining products and then leaving without buying; the online team saw rising traffic from people comparing prices. The two observations were never brought together, and the prevailing belief was not tested. By the time the pattern was recognised, the retailer had lost much of its market. The causes match those in the government cases: a fixed belief, information held in separate units and no one assigned to challenge the view.

Lessons for organisations

Betts (2007) cautions that reforms cannot eliminate failure and that some proposed remedies create new problems. Still, the cases suggest several practices that reduce the risk:

These lessons apply directly to business. Companies are surprised by competitors, by shifts in customer behaviour and by failures in their own operations, often for the same reasons: a fixed view of the market, the assumption that rivals think alike, and information held in separate departments.

Conclusion

The history of intelligence failure is not mainly a history of missing information. It is a history of information misread, unshared or ignored because it did not fit what people expected. The most useful lessons concern habits of mind and organisation: testing assumptions, considering alternatives, encouraging challenge and connecting information across boundaries. No organisation can avoid every surprise, but those that learn from the failures of others can avoid repeating them.

References

Allison, G., & Zelikow, P. (1999). Essence of decision: Explaining the Cuban missile crisis (2nd ed.). Longman.

Bar-Joseph, U. (2005). The watchman fell asleep: The surprise of Yom Kippur and its sources. State University of New York Press.

Betts, R. K. (1978). Analysis, war, and decision: Why intelligence failures are inevitable. World Politics, 31(1), 61–89. https://doi.org/10.2307/2009967

Betts, R. K. (2007). Enemies of intelligence: Knowledge and power in American national security. Columbia University Press.

Butler, R. (2004). Review of intelligence on weapons of mass destruction (HC 898). The Stationery Office.

Commission on the Intelligence Capabilities of the United States Regarding Weapons of Mass Destruction. (2005). Report to the President of the United States. U.S. Government Printing Office.

Heuer, R. J., Jr. (1999). Psychology of intelligence analysis. Center for the Study of Intelligence, Central Intelligence Agency.

Jervis, R. (2010). Why intelligence fails: Lessons from the Iranian Revolution and the Iraq War. Cornell University Press.

Kam, E. (1988). Surprise attack: The victim's perspective. Harvard University Press.

Knorr, K. (1964). Failures in national intelligence estimates: The case of the Cuban missiles. World Politics, 16(3), 455–467. https://doi.org/10.2307/2009582

National Commission on Terrorist Attacks Upon the United States. (2004). The 9/11 Commission report. W. W. Norton.

Pillar, P. R. (2011). Intelligence and U.S. foreign policy: Iraq, 9/11, and misguided reform. Columbia University Press.

Vaughan, D. (1996). The Challenger launch decision: Risky technology, culture, and deviance at NASA. University of Chicago Press.

Wohlstetter, R. (1962). Pearl Harbor: Warning and decision. Stanford University Press.

Zegart, A. B. (2007). Spying blind: The CIA, the FBI, and the origins of 9/11. Princeton University Press.