Investigation · Method and the future
Failure Analysis of Incomplete or Inconclusive Investigations
Many investigations end without a clear answer. Some reach the limits of the available evidence; others fail because of avoidable errors. This article explains how to tell the difference, the recurring causes of investigative failure, lessons from intelligence and criminal justice failures, methods for analysing what went wrong, and how to report an inconclusive result honestly.
Not every investigation ends with a clear answer. A fraud investigation may establish that money is missing without proving who took it. A workplace complaint may come down to one person's word against another's. A safety investigation may narrow the cause of an incident to two possibilities without being able to choose between them. A criminal case may remain unsolved for decades. Some of these outcomes are unavoidable: the evidence needed simply does not exist. Others are failures: the evidence existed but was not found, was mishandled, or was misread. Distinguishing between these is essential, both to learn from failure and to avoid blaming investigators for limits they could not overcome. This article explains how to analyse incomplete and inconclusive investigations, the recurring causes of failure, and how to report an uncertain outcome honestly.
Inconclusive is not the same as failed
An inconclusive result can be the correct result. If the available evidence does not establish what happened to the required standard, the honest finding is that the matter is not proved. Reaching a definite conclusion that the evidence does not support would be a worse failure. The question for failure analysis is therefore not "why did we not get an answer?" but "did we do what could reasonably have been done, and was the conclusion we reached the one the evidence supported?"
Recurring causes of failure
Studies of investigative failures, in criminal justice, intelligence and safety, identify recurring causes.
Tunnel vision and confirmation bias. The most frequently cited cause is premature focus on a single explanation or suspect. Findley and Scott (2006) describe tunnel vision as a product of natural cognitive tendencies, institutional pressures and the structure of adversarial systems, leading investigators to select and interpret evidence that supports their chosen account and to overlook or dismiss evidence pointing elsewhere. Nickerson (1998) reviews the extensive evidence that people seek and interpret information in ways that confirm existing beliefs. Fahsing and Ask (2013) found that detectives recognised "tipping points" after which alternative hypotheses received less attention. Rossmo (2009) collected case studies showing how such biases, along with organisational pressures and probability errors, have caused major investigations to fail.
Information overload and poor information management. The investigation of the "Yorkshire Ripper" murders in England failed for years in part because the incident room's paper records were so voluminous that connections between items could not be found; the offender was interviewed several times without being identified (Byford, 1981).
Flawed evidence and expert error. Garrett (2011), analysing the first 250 people exonerated by DNA testing in the United States, found that eyewitness misidentification, invalid forensic testimony, false confessions and unreliable informants recurred in the cases. The misidentification of Brandon Mayfield's fingerprint showed how experts can verify each other's errors when their reviews are not independent (Office of the Inspector General, 2006). Kassin et al. (2013) describe how contextual information can bias forensic judgements.
Organisational and cultural failures. The inquiry into the investigation of the murder of Stephen Lawrence found that the investigation had been undermined by incompetence, institutional racism and a failure of leadership (Macpherson, 1999). The Hillsborough Independent Panel found that police statements had been altered and that responsibility had been wrongly placed on victims (Hillsborough Independent Panel, 2012). Such failures are not errors of individual judgement alone; they reflect the culture and incentives of the organisation.
Normalisation of deviance. Vaughan (1996), analysing the decision to launch the space shuttle Challenger, described how small departures from standards became accepted over time as normal, until a catastrophic failure occurred. The Columbia Accident Investigation Board (2003) found similar organisational patterns behind the later loss of Columbia. In investigations, shortcuts in evidence handling, review or documentation can become normal practice in the same way.
When the answer was wrong
Failure analysis should also cover investigations that reached a definite conclusion that later proved wrong. These are in some ways worse than inconclusive outcomes, because they impose consequences on the wrong people and close the search for the truth. Wrongful convictions are the clearest example. Garrett (2011) found that in many of the DNA exoneration cases he studied, the original investigations had produced apparently strong evidence, such as confident eyewitness identifications, detailed confessions and forensic testimony, that was in fact unreliable. Findley and Scott (2006) argue that tunnel vision explains how such evidence accumulates: once investigators settle on a suspect, evidence that fits is sought out and accepted, while evidence that does not fit is explained away, so that the case appears stronger as it grows.
Wrong answers also occur outside criminal justice. A workplace investigation may dismiss an employee on the basis of a misread log; an audit may attribute a loss to fraud when it was caused by a system error; a safety investigation may blame an operator for a failure caused by a design flaw. In each case, the failure analysis should ask not only which evidence was misread but why the investigation's checks, such as peer review, testing of alternatives and the right of the person to respond, did not catch the error.
Lessons from intelligence failures
Intelligence analysis offers detailed studies of how investigations of a kind can reach wrong conclusions. Reviews of intelligence on Iraqi weapons of mass destruction before the 2003 invasion found that analysts had relied on a small number of unreliable sources, had not adequately tested their assumptions, and had presented uncertain judgements with too much confidence (Butler, 2004; Commission on the Intelligence Capabilities of the United States Regarding Weapons of Mass Destruction, 2005). Jervis (2010) argued that intelligence failures often arise not from lack of information but from the plausibility of a prevailing view, which leads analysts to interpret ambiguous evidence as confirming it, and from failure to state and test key assumptions. Heuer (1999) developed analysis of competing hypotheses partly to counter these tendencies, by requiring analysts to evaluate evidence against all reasonable hypotheses rather than only the favoured one.
The trap of hindsight
Failure analysis has its own biases. Once the outcome is known, it is easy to see which evidence mattered and to assume investigators should have seen it too. Fischhoff (1975) showed that knowing an outcome makes people believe it was more predictable than it was. Baron and Hershey (1988) found that people judge decisions as worse when they lead to bad outcomes, even when the decision process was identical. Dekker (2014) argues that understanding failure requires reconstructing what people knew, what they were trying to do and what pressures they faced at the time, rather than judging their actions against knowledge they did not have.
Reason (1997, 2000) distinguished between active failures, the errors of individuals at the sharp end, and latent conditions, such as inadequate procedures, training, resources or culture, that make such errors likely. Failure analyses that stop at individual error leave latent conditions in place to cause the next failure.
Methods of failure analysis
A structured failure analysis of an investigation typically involves:
- Reconstructing the timeline. Using the case record, establish what was known, when, and what decisions were made in response. Where the record is incomplete, interviews with the investigators can fill gaps, but their recollections should be checked against documents, because memory of a past case is itself shaped by its outcome.
- Identifying critical points. Find the decisions or events where a different choice could have changed the outcome.
- Assessing each decision in context. Ask whether the decision was reasonable given the information, resources and procedures available at the time.
- Mapping causes. Use tools such as cause-and-effect diagrams to consider causes across categories, including evidence, reasoning, people, methods, information management and organisation (Ishikawa, 1990).
- Distinguishing limits from failures. For each gap in the outcome, decide whether it reflects an unavoidable limit of the evidence or an avoidable failure.
- Identifying latent conditions. Look beyond individual errors to the conditions that made them likely.
- Recommending changes. Propose specific changes to procedures, training, tools, resources or culture, with owners.
Peerally et al. (2017) warn that root cause analyses often search for a single cause and produce weak recommendations. Investigative failures usually have several interacting causes, and analyses should resist the temptation to find one.
Supporting investigators after failure
How an organisation treats investigators after a failure determines whether it learns. If failure reviews are used mainly to discipline individuals, investigators will hide problems, avoid difficult cases and resist review. Edmondson (1999) found that teams learn more when members feel psychologically safe to admit mistakes and raise concerns. Failure analysis should therefore separate the question of what the organisation can learn from any question of individual misconduct, which should be dealt with through a separate process and only where there is evidence of recklessness or bad faith. Investigators whose cases are reviewed should be involved in the analysis, given the chance to explain their decisions, and told the outcome. Organisations should also recognise that investigators who have worked on failed or harrowing cases may need support.
Reopening an inconclusive investigation
An inconclusive investigation should be closed in a way that allows it to be reopened. The closing report should list the hypotheses that remain open, the evidence that would distinguish between them, and any material that could be re-examined with new methods. Organisations should set out what will trigger a reopening, such as new information from a witness, a link to a later case, or the availability of a new forensic technique. In criminal cases, advances in DNA analysis have allowed many cases closed as unsolved to be resolved years later, but only where exhibits were preserved and records were kept. A failure analysis that identifies what was missing gives a future reviewer a head start.
Tracking inconclusive outcomes
Organisations rarely measure how many of their investigations end without a clear answer, or why. Recording the outcome of every investigation, including whether it was inconclusive and the main reason, allows patterns to be seen. If many investigations fail because evidence was not preserved in time, the solution is a preservation procedure, not better individual effort. If many fail because key witnesses would not cooperate, the problem may be trust in the process. If inconclusive outcomes cluster in one team or type of case, that may point to resources, training or methods. Periodic thematic reviews of inconclusive cases, comparing them with successful ones, can reveal latent conditions that no single case review would detect.
Questions for a failure review
A review team can use a standard set of questions:
- What was the investigation trying to establish, and to what standard?
- What evidence was available, and what evidence existed but was not obtained?
- When was each key item of evidence sought, and was it still available?
- What hypotheses were considered, and when were any set aside?
- Were key analyses validated and independently reviewed?
- Were the people involved given a chance to respond?
- What pressures of time, workload, authority or expectation affected decisions?
- Which gaps reflect limits of the evidence, and which were avoidable?
- What conditions made the avoidable failures likely, and what will change them?
Reporting an inconclusive result
An inconclusive investigation still owes its audience a clear report. The report should state the questions, the evidence gathered, the hypotheses considered and how each was tested, and why the evidence does not allow a finding to the required standard. It should say which explanations remain possible and which have been ruled out, what evidence would resolve the matter if it became available, and whether any failures contributed to the outcome. This allows decision-makers to act on what is known, protects people from unfounded findings, and makes later review or reopening possible.
Inconclusive outcomes are hard on the people involved. A complainant may feel disbelieved, and a person who was investigated may feel that suspicion has not been lifted. The report and any communication should explain plainly that "not proved" means the evidence did not reach the required standard, not that the complaint was false or that the person was guilty. Where the organisation can still act without a finding against anyone, for example by changing a process, separating colleagues or improving controls, it should say so. Decision-makers should resist pressure to convert an inconclusive result into a definite one, in either direction, because doing so undermines confidence in every later investigation.
Weick and Sutcliffe (2007) describe "high-reliability organisations" as those that are preoccupied with failure, reluctant to simplify, and responsive to weak signals of trouble. An investigative organisation that treats inconclusive outcomes as opportunities to learn, rather than as embarrassments to be filed away, develops the same qualities.
A short example
The following hypothetical case illustrates failure analysis. A company investigates the disappearance of a batch of laptops from a storeroom and ends without identifying who took them. Six months later, a review team analyses the investigation. It reconstructs the timeline from the case record and finds three critical points. First, the storeroom's access logs were requested a week after the loss, by which time they had been overwritten under a routine retention setting; this was an avoidable failure caused by the absence of a preservation procedure. Second, the investigator focused early on a contractor who had been seen near the storeroom, and did not interview the two employees who held keys until a month later; the review finds this reflected tunnel vision but also a heavy caseload that left no time for parallel enquiries. Third, no camera covered the storeroom door, a limit that no investigative action could have overcome.
The review concludes that the investigation was inconclusive partly because of genuine limits and partly because of avoidable failures. Its recommendations address latent conditions: a standard preservation request sent within 24 hours of any loss, a requirement to list and test alternative explanations at the first case review, a workload limit for investigators, and a camera over the storeroom door. It does not recommend disciplinary action against the investigator, whose decisions were understandable given the conditions at the time.
Conclusion
Investigations that end without a clear answer are common, and an honest inconclusive finding is better than a confident wrong one. Failure analysis aims to distinguish unavoidable limits of the evidence from avoidable failures, and to identify the latent conditions behind those failures. Recurring causes include tunnel vision, information overload, flawed evidence, organisational culture and the gradual normalisation of shortcuts. Analyses must guard against hindsight and outcome bias and avoid stopping at individual error. Reports of inconclusive investigations should state what was done, what remains possible and what would resolve the matter. Organisations that study their inconclusive cases as carefully as their successes learn the lessons that prevent future failures.
References
Baron, J., & Hershey, J. C. (1988). Outcome bias in decision evaluation. Journal of Personality and Social Psychology, 54(4), 569–579. https://doi.org/10.1037/0022-3514.54.4.569
Butler, R. (2004). Review of intelligence on weapons of mass destruction: Report of a committee of Privy Counsellors (HC 898). The Stationery Office.
Byford, L. (1981). The Yorkshire Ripper case: Review of the police investigation of the case. Home Office.
Columbia Accident Investigation Board. (2003). Report (Vol. 1). National Aeronautics and Space Administration.
Commission on the Intelligence Capabilities of the United States Regarding Weapons of Mass Destruction. (2005). Report to the President of the United States.
Dekker, S. (2014). The field guide to understanding 'human error' (3rd ed.). Ashgate.
Edmondson, A. (1999). Psychological safety and learning behavior in work teams. Administrative Science Quarterly, 44(2), 350–383. https://doi.org/10.2307/2666999
Fahsing, I., & Ask, K. (2013). Decision making and decisional tipping points in homicide investigations: An interview study of British and Norwegian detectives. Journal of Investigative Psychology and Offender Profiling, 10(2), 155–165. https://doi.org/10.1002/jip.1384
Findley, K. A., & Scott, M. S. (2006). The multiple dimensions of tunnel vision in criminal cases. Wisconsin Law Review, 2006(2), 291–397.
Fischhoff, B. (1975). Hindsight ≠ foresight: The effect of outcome knowledge on judgment under uncertainty. Journal of Experimental Psychology: Human Perception and Performance, 1(3), 288–299. https://doi.org/10.1037/0096-1523.1.3.288
Garrett, B. L. (2011). Convicting the innocent: Where criminal prosecutions go wrong. Harvard University Press.
Heuer, R. J., Jr. (1999). Psychology of intelligence analysis. Center for the Study of Intelligence, Central Intelligence Agency.
Hillsborough Independent Panel. (2012). Hillsborough: The report of the Hillsborough Independent Panel (HC 581). The Stationery Office.
Ishikawa, K. (1990). Introduction to quality control. 3A Corporation.
Jervis, R. (2010). Why intelligence fails: Lessons from the Iranian revolution and the Iraq war. Cornell University Press.
Kassin, S. M., Dror, I. E., & Kukucka, J. (2013). The forensic confirmation bias: Problems, perspectives, and proposed solutions. Journal of Applied Research in Memory and Cognition, 2(1), 42–52. https://doi.org/10.1016/j.jarmac.2013.01.001
Macpherson, W. (1999). The Stephen Lawrence inquiry: Report of an inquiry by Sir William Macpherson of Cluny (Cm 4262-I). The Stationery Office.
Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2), 175–220. https://doi.org/10.1037/1089-2680.2.2.175
Office of the Inspector General. (2006). A review of the FBI's handling of the Brandon Mayfield case. U.S. Department of Justice.
Peerally, M. F., Carr, S., Waring, J., & Dixon-Woods, M. (2017). The problem with root cause analysis. BMJ Quality & Safety, 26(5), 417–422. https://doi.org/10.1136/bmjqs-2016-005511
Reason, J. (2000). Human error: Models and management. BMJ, 320(7237), 768–770. https://doi.org/10.1136/bmj.320.7237.768
Reason, J. (1997). Managing the risks of organizational accidents. Ashgate.
Rossmo, D. K. (2009). Criminal investigative failures. CRC Press.
Vaughan, D. (1996). The Challenger launch decision: Risky technology, culture, and deviance at NASA. University of Chicago Press.
Weick, K. E., & Sutcliffe, K. M. (2007). Managing the unexpected: Resilient performance in an age of uncertainty (2nd ed.). Jossey-Bass.