Skip to content

Investigation · Method and the future

Simulation and Tabletop Exercises for Testing Investigative Knowledge Processes

Exercises let investigation teams test their plans, procedures and decision-making before a real case does it for them. This article explains the types of exercise, from tabletop discussions to full simulations, the evidence that simulation improves performance, how to design and run an exercise, and how to turn its findings into improvements.

ModQu research article · 913 words · 11 references (APA 7th edition)

Download PDF

Most organisations discover the weaknesses in their investigation processes during a real investigation, when the cost of a gap is highest. Exercises allow those weaknesses to be found in advance. A tabletop exercise walks a team through a realistic scenario in discussion; a full simulation has people carry out their roles in real time, with systems, evidence and pressure. Both test whether plans work, whether people know their roles, whether information flows to the right place and whether decisions can be made under uncertainty. This article explains the main types of exercise, the evidence for their value and how to use them well.

Types of exercise

Table of five types of exercise: seminar or workshop, tabletop exercise, drill, functional exercise and full-scale simulation, with what happens and what each suits.
Figure 1. Exercises range from discussion-based to operations-based. Most organisations start with tabletop exercises and move to more realistic simulations as their procedures mature. Diagram by ModQu.

Emergency management guidance distinguishes discussion-based exercises, such as seminars and tabletop exercises, from operations-based exercises, such as drills, functional exercises and full-scale exercises (Federal Emergency Management Agency [FEMA], 2020). Guidance for information technology plans uses similar categories and recommends a programme that progresses from simpler to more complex exercises (Grance et al., 2006). Guidance on computer security incident handling recommends exercising incident response plans, including the investigation and evidence-handling steps, before they are needed (Cichonski et al., 2012).

Evidence for simulation

Simulation-based training is well established in fields where errors are costly. In medicine, systematic reviews have found that simulation improves knowledge, skills and behaviour, especially when it includes feedback, repeated practice and a range of difficulty (Issenberg et al., 2005; Cook et al., 2011). Salas et al. (2009) emphasise that the value of simulation depends on measuring performance against clear objectives and giving structured feedback. Debriefing after exercises is particularly important: Tannenbaum and Cerasoli (2013) found that structured debriefs improve team performance substantially.

In policing, immersive simulations have been used to train and study senior investigators managing critical incidents, such as kidnappings and major crimes. Crego and Alison (2004) used such simulations to study how officers perceived critical decisions, and Alison et al. (2013) reviewed how immersive environments allow realistic pressure and decision-making to be studied and trained safely.

Exercises can also expose problems that are not then fixed. A 2004 exercise in the United States simulated a catastrophic hurricane striking New Orleans and identified many of the problems that later occurred during Hurricane Katrina; a congressional inquiry found that lessons from the exercise had not been fully acted on (U.S. House of Representatives, 2006).

Designing an exercise

A useful exercise starts with clear objectives, such as testing whether evidence is preserved within set times or whether a whistleblowing report reaches the right person. The scenario should be realistic and drawn from the organisation's own risks, with injects, new pieces of information released during the exercise, that force decisions. Participants should include everyone with a role in a real investigation, such as investigators, information technology staff, lawyers, human resources, communications and senior managers. A facilitator controls the pace, and observers record what happens against the objectives. A "premortem", in which participants imagine the investigation has failed and explain why, can be a quick way to surface risks (Klein, 2007).

Running and evaluating the exercise

During the exercise, the facilitator should keep participants in role, release injects at a realistic pace and resist the temptation to rescue the team when it struggles, because struggle is where gaps appear. Observers should record decisions, the time they took, the information used and any points of confusion, against an evaluation guide prepared in advance. Emergency management guidance recommends a short "hot wash" immediately afterwards, while impressions are fresh, followed by a written after-action report and an improvement plan that assigns each corrective action to an owner with a deadline (FEMA, 2020). The exercise should be judged against its objectives rather than by whether the scenario was "won": an exercise that exposes several gaps has done its job.

Common mistakes

Exercises lose their value when the scenario is too easy, when the people who would actually make decisions send deputies, when participants are told the answers in advance, or when the exercise is treated as a test of individuals rather than of the process. Another common mistake is to run an exercise once and then file the report. The improvement plan should be tracked, and a later exercise should test whether the changes worked. Exercises should also be varied over time to cover different types of incident, such as fraud, data leaks, harassment complaints and safety failures, and to include scenarios in which the obvious explanation turns out to be wrong.

A short example

The following hypothetical case shows a tabletop exercise in practice. A survey company runs a two-hour exercise on a suspected leak of respondent data by an employee. As the facilitator releases injects, the team finds that no one knows who can authorise preserving the employee's mailbox, that the information technology team would wipe the laptop as part of routine reissue, and that the communications team would learn of the leak only from the press. The debrief produces three actions: a named authoriser for preservation, a hold procedure for devices, and a contact list for incidents. A follow-up exercise six months later confirms that all three work.

Conclusion

Exercises test investigative processes before a real case does. Tabletop exercises are cheap and effective for testing roles and decisions; simulations add realism and pressure. Their value depends on clear objectives, realistic scenarios, structured debriefing and, above all, acting on what they reveal.

References

Alison, L., van den Heuvel, C., Waring, S., Power, N., Long, A., O'Hara, T., & Crego, J. (2013). Immersive simulated learning environments for researching critical incidents: A knowledge synthesis of the literature and experiences of studying high-risk strategic decision making. Journal of Cognitive Engineering and Decision Making, 7(3), 255–272. https://doi.org/10.1177/1555343412468113

Cichonski, P., Millar, T., Grance, T., & Scarfone, K. (2012). Computer security incident handling guide (NIST Special Publication 800-61 Revision 2). National Institute of Standards and Technology. https://doi.org/10.6028/NIST.SP.800-61r2

Cook, D. A., Hatala, R., Brydges, R., Zendejas, B., Szostek, J. H., Wang, A. T., Erwin, P. J., & Hamstra, S. J. (2011). Technology-enhanced simulation for health professions education: A systematic review and meta-analysis. JAMA, 306(9), 978–988. https://doi.org/10.1001/jama.2011.1234

Crego, J., & Alison, L. (2004). Control and legacy as functions of perceived criticality in major incidents. Journal of Investigative Psychology and Offender Profiling, 1(3), 207–225. https://doi.org/10.1002/jip.14

Federal Emergency Management Agency. (2020). Homeland Security Exercise and Evaluation Program (HSEEP). U.S. Department of Homeland Security.

Grance, T., Nolan, T., Burke, K., Dudley, R., White, G., & Good, T. (2006). Guide to test, training, and exercise programs for IT plans and capabilities (NIST Special Publication 800-84). National Institute of Standards and Technology. https://doi.org/10.6028/NIST.SP.800-84

Issenberg, S. B., McGaghie, W. C., Petrusa, E. R., Gordon, D. L., & Scalese, R. J. (2005). Features and uses of high-fidelity medical simulations that lead to effective learning: A BEME systematic review. Medical Teacher, 27(1), 10–28. https://doi.org/10.1080/01421590500046924

Klein, G. (2007). Performing a project premortem. Harvard Business Review, 85(9), 18–19.

Salas, E., Rosen, M. A., Held, J. D., & Weissmuller, J. J. (2009). Performance measurement in simulation-based training: A review and best practices. Simulation & Gaming, 40(3), 328–376. https://doi.org/10.1177/1046878108326734

Tannenbaum, S. I., & Cerasoli, C. P. (2013). Do team and individual debriefs enhance performance? A meta-analysis. Human Factors, 55(1), 231–245. https://doi.org/10.1177/0018720812448394

U.S. House of Representatives. (2006). A failure of initiative: Final report of the Select Bipartisan Committee to Investigate the Preparation for and Response to Hurricane Katrina (H. Rept. 109-377).