Intelligence
Intelligence glossary
Terms from intelligence analysis and business, market, competitive and open-source intelligence, including forecasting and analytic tradecraft.
A
- All-source intelligence also: all-source fusion
Intelligence produced by combining information from every available collection discipline, such as human sources, signals, imagery and open sources, into one assessment. Fusing sources allows analysts to corroborate reports and compensate for the gaps and biases of any single source. (Kent, 1949)
- Analysis of competing hypotheses also: ACH
A structured analytic technique developed by Richards Heuer in which analysts list all plausible hypotheses, array the evidence against each in a matrix, and focus on disconfirming evidence: the most likely hypothesis is the one with the least evidence against it, not the most evidence for it. It is designed to counter confirmation bias and premature closure. (Heuer, 1999)
B
- Brier score
A measure of the accuracy of probability forecasts, equal to the mean squared difference between the forecast probabilities and the outcomes (1 if the event happened, 0 if not). Lower is better. It can be decomposed into calibration and resolution, and it was the main scoring rule in the forecasting tournaments that identified superforecasters. (Brier, 1950; Murphy, 1973)
- Business intelligence also: BI
The processes, technologies and practices an organisation uses to collect, integrate, analyse and present data about its operations and environment in support of decisions. Hans Peter Luhn used the term in 1958 for an automated system to disseminate information; it now usually refers to data warehouses, reporting and dashboards. (Luhn, 1958; Chen et al., 2012)
C
- Calibration
The match between stated probabilities and observed frequencies: a forecaster is well calibrated if events given a 70% chance happen about 70% of the time. Calibration can be assessed with calibration curves and the calibration component of the Brier score, and it improves with training and feedback. (Murphy, 1973; Mandel & Barnes, 2014)
- Collective intelligence
The capacity of a group to perform well across a range of tasks, beyond the abilities of its individual members. Research has found a general collective intelligence factor in small groups that is more strongly related to members' social sensitivity and equality of conversational turn-taking than to the average or maximum intelligence of members. (Woolley et al., 2010)
- Competitive intelligence also: CI
The legal and ethical collection and analysis of information about competitors, markets and the business environment to support strategic and tactical decisions. It differs from industrial espionage in relying on public sources, interviews and observation rather than deception or theft. (Porter, 1979)
- Counterintelligence
Activities to identify, deter and counter efforts by others to obtain an organisation's protected information or to deceive it. In business it covers protecting trade secrets and strategic plans from competitors, insiders and foreign intelligence services. (Kent, 1949)
D
- Dashboard
A visual display of the most important information needed to achieve one or more objectives, arranged on a single screen so it can be monitored at a glance. Good dashboards show a small number of decision-relevant indicators with context, such as targets and trends, rather than decoration. (Tufte, 1983; Chen et al., 2012)
- Deception also: denial and deception
Deliberate efforts by an adversary to mislead an analyst or decision-maker by concealing the truth (denial) or presenting falsehoods (deception), so that the target acts against its own interests. Analysts counter it by checking source access, looking for anomalies and testing whether evidence could have been planted. (Heuer, 1999)
- Descriptive, predictive and prescriptive analytics
A common classification of analytics by the question asked: descriptive (what happened), diagnostic (why it happened), predictive (what is likely to happen) and prescriptive (what should be done). Each stage requires more modelling and stronger assumptions than the one before. (Chen et al., 2012)
- Disinformation
False information deliberately created and spread to deceive or cause harm. It is distinguished from misinformation, false information shared without intent to deceive, and malinformation, genuine information shared to cause harm. False news has been found to spread farther and faster online than true news. (Wardle & Derakhshan, 2017; Vosoughi et al., 2018)
F
- Five forces also: Porter's five forces
Michael Porter's framework for analysing the competitive structure of an industry through five forces: rivalry among existing competitors, the threat of new entrants, the threat of substitutes, and the bargaining power of buyers and of suppliers. The combined strength of the forces shapes the industry's profit potential. (Porter, 1979)
G
- Groupthink
Irving Janis's term for a mode of thinking in cohesive groups where the desire for agreement overrides realistic appraisal of alternatives. Symptoms include an illusion of invulnerability, self-censorship, pressure on dissenters and stereotyped views of outsiders. Devil's advocacy, red teams and independent reviews are common remedies. (Janis, 1972)
H
- Hallucination
Output from an AI language model that is fluent and confident but not supported by its sources or by fact, such as an invented citation. Because hallucinations are hard to distinguish from accurate output, AI-assisted research and intelligence work requires verification of every factual claim against primary sources. (Ji et al., 2023; Bender et al., 2021)
I
- Indicators and warnings also: I&W
A method of monitoring for observable signs that a specified development, such as an attack, a market shift or a supplier failure, is becoming more likely. Analysts define in advance which indicators would signal change and track them systematically, which reduces the risk of surprise. (Heuer, 1999; Ansoff, 1975)
- Information overload
The state in which the amount of information exceeds a person's or organisation's capacity to process it, so that decision quality falls rather than rises with more information. Causes include volume, poor quality and poor organisation of information; remedies include filtering, prioritisation and better presentation. (Eppler & Mengis, 2004)
- Intelligence
In the organisational sense, information that has been collected, evaluated and analysed to give a decision-maker an advantage, together with the organisation and activity that produce it. Sherman Kent described intelligence as knowledge, organisation and activity. It differs from raw information in being evaluated and tailored to a decision. (Kent, 1949)
- Intelligence cycle
The conventional model of the intelligence process as a cycle of direction (setting requirements), collection, processing, analysis and production, and dissemination, followed by feedback. Critics argue that real intelligence work is less sequential, with analysts and policymakers interacting throughout. (Hulnick, 2006; Kent, 1949)
- Intelligence failure
A failure to anticipate or correctly assess a significant development, such as the attack on Pearl Harbor. Studies emphasise that failures usually stem less from missing information than from ambiguous signals, mistaken assumptions, organisational barriers and decision-makers' reluctance to act on warning, which is why some failures are unavoidable. (Betts, 1978)
- Intelligence requirement also: key intelligence question
A statement of the information a decision-maker needs, expressed as a question that collection and analysis can address. Key intelligence questions or topics prioritise effort, link the intelligence function to decisions and give a basis for measuring its value. (Hulnick, 2006)
K
- Key assumptions check
A structured technique in which analysts list the assumptions underlying a judgement, ask how confident they are in each and what would make it untrue, and identify which ones the conclusion depends on. It brings hidden premises into the open before they cause a failure. (Heuer, 1999)
- Key performance indicator also: KPI
A measure chosen to track progress towards an important objective, such as customer retention or cost per acquisition. Good KPIs are few, clearly defined, linked to strategy and balanced, as in the balanced scorecard's financial, customer, internal process and learning perspectives. Targets set on a measure can distort behaviour. (Kaplan & Norton, 1992)
M
- Market intelligence
Information and analysis about a market's size, growth, segments, customers, competitors, prices and trends, gathered from primary research, secondary sources and internal data to inform market entry, product and strategy decisions. (Chen et al., 2012)
O
- Open-source intelligence also: OSINT
Intelligence derived from publicly available information, such as media, public records, company filings, academic work, satellite imagery and social media. The growth of the internet has made it the largest source for many questions; its main challenges are volume, verification and deception. (Glassman & Kang, 2012)
- Organisational intelligence
An organisation's capacity to gather, interpret and use information about its environment to make decisions. Harold Wilensky's 1967 study showed how hierarchy, specialisation and secrecy distort the flow of intelligence in government and industry. (Wilensky, 1967; Huber, 1991)
P
- Prediction market
A market in which participants trade contracts whose payoff depends on a future event, so that prices can be read as aggregate probability estimates. Prediction markets have forecast elections and project outcomes well, though in forecasting tournaments well-aggregated team forecasts have matched or beaten them. (Wolfers & Zitzewitz, 2004; Atanasov et al., 2017)
R
- Red teaming also: devil's advocacy
Assigning a group to take the perspective of an adversary or to challenge a plan, so that weaknesses and alternative outcomes are exposed before they are exploited. Devil's advocacy, in which one person argues against the prevailing view, is a lighter form of the same idea. (Zenko, 2015; Heuer, 1999)
S
- Scenario planning
A method for exploring several plausible futures, built from the key uncertainties and driving forces affecting an organisation, so that strategies can be tested against each. Its aim is to widen thinking and prepare for change rather than to predict a single outcome. (Schoemaker, 1995)
- Source evaluation
Assessing information by the reliability of its source and the credibility of its content, often with a two-part grading such as the NATO Admiralty code (A–F for source reliability, 1–6 for information credibility). Research shows such ratings are often inconsistent and that analysts should consider a source's access and corroboration explicitly. (Irwin & Mandel, 2019)
- Structured analytic techniques also: SATs
Methods that make an analyst's reasoning explicit and open to review, such as analysis of competing hypotheses, key assumptions checks, red teaming, indicators and scenarios. They were promoted to counter the cognitive biases Heuer described, although evidence on how much they improve accuracy is mixed. (Heuer, 1999)
- Superforecasting
The practice of the top performers in the Good Judgment Project, a forecasting tournament run from 2011 to 2015, whose probability forecasts on geopolitical questions were markedly more accurate than those of other participants. Accuracy was linked to cognitive style, training in probabilistic reasoning, teamwork and frequent small updates. (Tetlock & Gardner, 2015; Mellers et al., 2015)
- SWOT analysis
A planning framework that lists an organisation's internal strengths and weaknesses and its external opportunities and threats. It is easy to use and communicate but is often criticised for producing unprioritised lists; it is most useful when each item is backed by evidence and linked to decisions. (Porter, 1979)
W
- Weak signal
An early, imprecise indication of a change that may become significant. Igor Ansoff argued in 1975 that firms should respond in graduated steps to weak signals of strategic surprise rather than wait for full information. Horizon scanning is the systematic search for such signals. (Ansoff, 1975)
- Wisdom of crowds
The finding that the aggregate of many independent judgements is often more accurate than most individual judgements, as in Francis Galton's 1907 report that the median guess of an ox's weight at a fair was within 1% of the true value. It depends on diversity and independence of opinions; when judgements are correlated or influenced by each other the advantage shrinks. (Galton, 1907; Surowiecki, 2004)
- Words of estimative probability also: WEP
Verbal expressions such as "likely" or "almost certain" used in intelligence estimates to convey probability. Sherman Kent showed in 1964 that readers interpret such words very differently and proposed linking each to a numerical range. Many intelligence agencies now publish standard probability yardsticks. (Kent, 1964; Mandel & Barnes, 2014)
References
Ansoff, H. I. (1975). Managing strategic surprise by response to weak signals. California Management Review, 18(2), 21–33. https://doi.org/10.2307/41164635
Atanasov, P., Rescober, P., Stone, E., Swift, S. A., Servan-Schreiber, E., Tetlock, P., Ungar, L., & Mellers, B. (2017). Distilling the wisdom of crowds: Prediction markets vs. prediction polls. Management Science, 63(3), 691–706. https://doi.org/10.1287/mnsc.2015.2374
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (pp. 610–623). ACM. https://doi.org/10.1145/3442188.3445922
Betts, R. K. (1978). Analysis, war, and decision: Why intelligence failures are inevitable. World Politics, 31(1), 61–89. https://doi.org/10.2307/2009967
Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1), 1–3. https://doi.org/10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2
Chen, H., Chiang, R. H. L., & Storey, V. C. (2012). Business intelligence and analytics: From big data to big impact. MIS Quarterly, 36(4), 1165–1188. https://doi.org/10.2307/41703503
Eppler, M. J., & Mengis, J. (2004). The concept of information overload: A review of literature from organization science, accounting, marketing, MIS, and related disciplines. The Information Society, 20(5), 325–344. https://doi.org/10.1080/01972240490507974
Galton, F. (1907). Vox populi. Nature, 75(1949), 450–451. https://doi.org/10.1038/075450a0
Glassman, M., & Kang, M. J. (2012). Intelligence in the internet age: The emergence and evolution of Open Source Intelligence (OSINT). Computers in Human Behavior, 28(2), 673–682. https://doi.org/10.1016/j.chb.2011.11.014
Heuer, R. J., Jr. (1999). Psychology of intelligence analysis. Center for the Study of Intelligence, Central Intelligence Agency.
Huber, G. P. (1991). Organizational learning: The contributing processes and the literatures. Organization Science, 2(1), 88–115. https://doi.org/10.1287/orsc.2.1.88
Hulnick, A. S. (2006). What's wrong with the intelligence cycle. Intelligence and National Security, 21(6), 959–979. https://doi.org/10.1080/02684520601046291
Irwin, D., & Mandel, D. R. (2019). Improving information evaluation for intelligence production. Intelligence and National Security, 34(4), 503–525. https://doi.org/10.1080/02684527.2019.1569343
Janis, I. L. (1972). Victims of groupthink: A psychological study of foreign-policy decisions and fiascoes. Houghton Mifflin.
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12), Article 248. https://doi.org/10.1145/3571730
Kaplan, R. S., & Norton, D. P. (1992). The balanced scorecard: Measures that drive performance. Harvard Business Review, 70(1), 71–79.
Kent, S. (1949). Strategic intelligence for American world policy. Princeton University Press.
Kent, S. (1964). Words of estimative probability. Studies in Intelligence, 8(4), 49–65.
Luhn, H. P. (1958). A business intelligence system. IBM Journal of Research and Development, 2(4), 314–319. https://doi.org/10.1147/rd.24.0314
Mandel, D. R., & Barnes, A. (2014). Accuracy of forecasts in strategic intelligence. Proceedings of the National Academy of Sciences, 111(30), 10984–10989. https://doi.org/10.1073/pnas.1406138111
Mellers, B., Stone, E., Murray, T., Minster, A., Rohrbaugh, N., Bishop, M., Chen, E., Baker, J., Hou, Y., Horowitz, M., Ungar, L., & Tetlock, P. (2015). Identifying and cultivating superforecasters as a method of improving probabilistic predictions. Perspectives on Psychological Science, 10(3), 267–281. https://doi.org/10.1177/1745691615577794
Murphy, A. H. (1973). A new vector partition of the probability score. Journal of Applied Meteorology, 12(4), 595–600. https://doi.org/10.1175/1520-0450(1973)012<0595:ANVPOT>2.0.CO;2
Porter, M. E. (1979). How competitive forces shape strategy. Harvard Business Review, 57(2), 137–145.
Schoemaker, P. J. H. (1995). Scenario planning: A tool for strategic thinking. Sloan Management Review, 36(2), 25–40.
Surowiecki, J. (2004). The wisdom of crowds. Doubleday.
Tetlock, P. E., & Gardner, D. (2015). Superforecasting: The art and science of prediction. Crown.
Tufte, E. R. (1983). The visual display of quantitative information. Graphics Press.
Vosoughi, S., Roy, D., & Aral, S. (2018). The spread of true and false news online. Science, 359(6380), 1146–1151. https://doi.org/10.1126/science.aap9559
Wardle, C., & Derakhshan, H. (2017). Information disorder: Toward an interdisciplinary framework for research and policy making (Council of Europe report DGI(2017)09). Council of Europe.
Wilensky, H. L. (1967). Organizational intelligence: Knowledge and policy in government and industry. Basic Books.
Wolfers, J., & Zitzewitz, E. (2004). Prediction markets. Journal of Economic Perspectives, 18(2), 107–126. https://doi.org/10.1257/0895330041371321
Woolley, A. W., Chabris, C. F., Pentland, A., Hashmi, N., & Malone, T. W. (2010). Evidence for a collective intelligence factor in the performance of human groups. Science, 330(6004), 686–688. https://doi.org/10.1126/science.1193147
Zenko, M. (2015). Red team: How to succeed by thinking like the enemy. Basic Books.