Intelligence

Intelligence glossary

Terms from intelligence analysis and business, market, competitive and open-source intelligence, including forecasting and analytic tradecraft.

A

All-source intelligence also: all-source fusion

Intelligence produced by combining information from every available collection discipline, such as human sources, signals, imagery and open sources, into one assessment. Fusing sources allows analysts to corroborate reports and compensate for the gaps and biases of any single source. (Kent, 1949)

Analysis and tradecraft
See also: Open-source intelligence, Corroboration (Investigation), Source evaluation
Read more: All-Source Intelligence Fusion Methodologies for Comprehensive Knowledge Integration

Analysis of competing hypotheses also: ACH

A structured analytic technique developed by Richards Heuer in which analysts list all plausible hypotheses, array the evidence against each in a matrix, and focus on disconfirming evidence: the most likely hypothesis is the one with the least evidence against it, not the most evidence for it. It is designed to counter confirmation bias and premature closure. (Heuer, 1999)

Analysis and tradecraft
See also: Structured analytic techniques, Confirmation bias (Insights), Key assumptions check
Read more: Analysis of Competing Hypotheses

B

Brier score

A measure of the accuracy of probability forecasts, equal to the mean squared difference between the forecast probabilities and the outcomes (1 if the event happened, 0 if not). Lower is better. It can be decomposed into calibration and resolution, and it was the main scoring rule in the forecasting tournaments that identified superforecasters. (Brier, 1950; Murphy, 1973)

Analysis and tradecraft
See also: Calibration, Superforecasting
Read more: Superforecasting: What Research Says About Accurate Prediction

Business intelligence also: BI

The processes, technologies and practices an organisation uses to collect, integrate, analyse and present data about its operations and environment in support of decisions. Hans Peter Luhn used the term in 1958 for an automated system to disseminate information; it now usually refers to data warehouses, reporting and dashboards. (Luhn, 1958; Chen et al., 2012)

Business intelligence
See also: Key performance indicator, Dashboard, Descriptive, predictive and prescriptive analytics
Read more: Business Intelligence: Systems, Processes and Value · A History of Business Intelligence From Ledgers to Dashboards

C

Calibration

The match between stated probabilities and observed frequencies: a forecaster is well calibrated if events given a 70% chance happen about 70% of the time. Calibration can be assessed with calibration curves and the calibration component of the Brier score, and it improves with training and feedback. (Murphy, 1973; Mandel & Barnes, 2014)

Analysis and tradecraft
See also: Brier score, Words of estimative probability
Read more: Superforecasting: What Research Says About Accurate Prediction · Expressing Probability in Intelligence Estimates: Words of Estimative Probability

Collective intelligence

The capacity of a group to perform well across a range of tasks, beyond the abilities of its individual members. Research has found a general collective intelligence factor in small groups that is more strongly related to members' social sensitivity and equality of conversational turn-taking than to the average or maximum intelligence of members. (Woolley et al., 2010)

Artificial and collective intelligence
See also: Wisdom of crowds, Prediction market
Read more: Collective Intelligence and the Wisdom of Crowds · Surveys as Instruments of Collective Intelligence

Competitive intelligence also: CI

The legal and ethical collection and analysis of information about competitors, markets and the business environment to support strategic and tactical decisions. It differs from industrial espionage in relying on public sources, interviews and observation rather than deception or theft. (Porter, 1979)

Market and competitive intelligence
See also: Market intelligence, Open-source intelligence, Five forces
Read more: Competitive Intelligence: Ethical Collection and Analysis · The Ethics and Legality of Competitive Intelligence

Counterintelligence

Activities to identify, deter and counter efforts by others to obtain an organisation's protected information or to deceive it. In business it covers protecting trade secrets and strategic plans from competitors, insiders and foreign intelligence services. (Kent, 1949)

Organisational intelligence
See also: Deception
Read more: Counterintelligence for Businesses: Protecting Against Espionage

D

Dashboard

A visual display of the most important information needed to achieve one or more objectives, arranged on a single screen so it can be monitored at a glance. Good dashboards show a small number of decision-relevant indicators with context, such as targets and trends, rather than decoration. (Tufte, 1983; Chen et al., 2012)

Business intelligence
See also: Key performance indicator, Business intelligence
Read more: Dashboards That Inform Decisions Rather Than Decorate Them

Deception also: denial and deception

Deliberate efforts by an adversary to mislead an analyst or decision-maker by concealing the truth (denial) or presenting falsehoods (deception), so that the target acts against its own interests. Analysts counter it by checking source access, looking for anomalies and testing whether evidence could have been planted. (Heuer, 1999)

Analysis and tradecraft
See also: Disinformation, Counterintelligence, Source evaluation
Read more: Deception and Disinformation: Counter-Narratives in Knowledge Generation

Descriptive, predictive and prescriptive analytics

A common classification of analytics by the question asked: descriptive (what happened), diagnostic (why it happened), predictive (what is likely to happen) and prescriptive (what should be done). Each stage requires more modelling and stronger assumptions than the one before. (Chen et al., 2012)

Business intelligence
See also: Business intelligence
Read more: Descriptive, Diagnostic, Predictive and Prescriptive Analytics

Disinformation

False information deliberately created and spread to deceive or cause harm. It is distinguished from misinformation, false information shared without intent to deceive, and malinformation, genuine information shared to cause harm. False news has been found to spread farther and faster online than true news. (Wardle & Derakhshan, 2017; Vosoughi et al., 2018)

Analysis and tradecraft
See also: Deception, Source evaluation
Read more: Deception and Disinformation: Counter-Narratives in Knowledge Generation · Investigating Disinformation Campaigns

F

Five forces also: Porter's five forces

Michael Porter's framework for analysing the competitive structure of an industry through five forces: rivalry among existing competitors, the threat of new entrants, the threat of substitutes, and the bargaining power of buyers and of suppliers. The combined strength of the forces shapes the industry's profit potential. (Porter, 1979)

Market and competitive intelligence
See also: Competitive intelligence, SWOT analysis
Read more: Porter's Five Forces as an Intelligence Framework

G

Groupthink

Irving Janis's term for a mode of thinking in cohesive groups where the desire for agreement overrides realistic appraisal of alternatives. Symptoms include an illusion of invulnerability, self-censorship, pressure on dissenters and stereotyped views of outsiders. Devil's advocacy, red teams and independent reviews are common remedies. (Janis, 1972)

Analysis and tradecraft
See also: Red teaming, Intelligence failure
Read more: Groupthink and Organisational Barriers to Good Intelligence

H

Hallucination

Output from an AI language model that is fluent and confident but not supported by its sources or by fact, such as an invented citation. Because hallucinations are hard to distinguish from accurate output, AI-assisted research and intelligence work requires verification of every factual claim against primary sources. (Ji et al., 2023; Bender et al., 2021)

Artificial and collective intelligence
See also: Source evaluation, Machine-generated knowledge (The Generation of Knowledge)
Read more: Risks of Automated Intelligence: Hallucination, Bias and Overconfidence · Large Language Models as Research and Intelligence Assistants

I

Indicators and warnings also: I&W

A method of monitoring for observable signs that a specified development, such as an attack, a market shift or a supplier failure, is becoming more likely. Analysts define in advance which indicators would signal change and track them systematically, which reduces the risk of surprise. (Heuer, 1999; Ansoff, 1975)

Analysis and tradecraft
See also: Weak signal, Scenario planning
Read more: Indicators and Warnings: Detecting Change Early

Information overload

The state in which the amount of information exceeds a person's or organisation's capacity to process it, so that decision quality falls rather than rises with more information. Causes include volume, poor quality and poor organisation of information; remedies include filtering, prioritisation and better presentation. (Eppler & Mengis, 2004)

Organisational intelligence
See also: Dashboard, Intelligence requirement
Read more: Information Overload and the Filtering Problem · Insight Fatigue and Information Overload in High-Volume Analytical Settings

Intelligence

In the organisational sense, information that has been collected, evaluated and analysed to give a decision-maker an advantage, together with the organisation and activity that produce it. Sherman Kent described intelligence as knowledge, organisation and activity. It differs from raw information in being evaluated and tailored to a decision. (Kent, 1949)

Concepts and history
See also: Intelligence cycle, Business intelligence
Read more: What Is Intelligence? Information, Analysis and Decision Advantage · Data, Information, Intelligence and Knowledge: Drawing the Distinctions · Sherman Kent and the Founding of Intelligence Analysis as a Discipline

Intelligence cycle

The conventional model of the intelligence process as a cycle of direction (setting requirements), collection, processing, analysis and production, and dissemination, followed by feedback. Critics argue that real intelligence work is less sequential, with analysts and policymakers interacting throughout. (Hulnick, 2006; Kent, 1949)

Concepts and history
See also: Intelligence requirement, Intelligence
Read more: The Intelligence Cycle: Direction, Collection, Processing, Analysis and Dissemination · Criticisms of the Intelligence Cycle and Alternative Models

Intelligence failure

A failure to anticipate or correctly assess a significant development, such as the attack on Pearl Harbor. Studies emphasise that failures usually stem less from missing information than from ambiguous signals, mistaken assumptions, organisational barriers and decision-makers' reluctance to act on warning, which is why some failures are unavoidable. (Betts, 1978)

Analysis and tradecraft
See also: Groupthink, Key assumptions check, Weak signal
Read more: Intelligence Failures: Case Studies and Lessons

Intelligence requirement also: key intelligence question

A statement of the information a decision-maker needs, expressed as a question that collection and analysis can address. Key intelligence questions or topics prioritise effort, link the intelligence function to decisions and give a basis for measuring its value. (Hulnick, 2006)

Organisational intelligence
See also: Intelligence cycle
Read more: Intelligence Requirements: Asking Decision-Makers What They Need to Know

K

Key assumptions check

A structured technique in which analysts list the assumptions underlying a judgement, ask how confident they are in each and what would make it untrue, and identify which ones the conclusion depends on. It brings hidden premises into the open before they cause a failure. (Heuer, 1999)

Analysis and tradecraft
See also: Analysis of competing hypotheses, Structured analytic techniques
Read more: Key Assumptions Checks

Key performance indicator also: KPI

A measure chosen to track progress towards an important objective, such as customer retention or cost per acquisition. Good KPIs are few, clearly defined, linked to strategy and balanced, as in the balanced scorecard's financial, customer, internal process and learning perspectives. Targets set on a measure can distort behaviour. (Kaplan & Norton, 1992)

Business intelligence
See also: Dashboard, Business intelligence
Read more: Key Performance Indicators: Selection, Design and Misuse

M

Market intelligence

Information and analysis about a market's size, growth, segments, customers, competitors, prices and trends, gathered from primary research, secondary sources and internal data to inform market entry, product and strategy decisions. (Chen et al., 2012)

Market and competitive intelligence
See also: Competitive intelligence
Read more: Market Intelligence: Understanding Size, Trends and Dynamics

O

Open-source intelligence also: OSINT

Intelligence derived from publicly available information, such as media, public records, company filings, academic work, satellite imagery and social media. The growth of the internet has made it the largest source for many questions; its main challenges are volume, verification and deception. (Glassman & Kang, 2012)

Open-source intelligence
See also: Source evaluation, All-source intelligence
Read more: Open-Source Intelligence (OSINT): Methods and Sources · OSINT for Business: Public Records, Filings and Media · Verifying Open-Source Information

Organisational intelligence

An organisation's capacity to gather, interpret and use information about its environment to make decisions. Harold Wilensky's 1967 study showed how hierarchy, specialisation and secrecy distort the flow of intelligence in government and industry. (Wilensky, 1967; Huber, 1991)

Organisational intelligence
See also: Double-loop learning (Inquiry), Intelligence
Read more: Organisational Intelligence: How Companies Learn and Sense Their Environment

P

Prediction market

A market in which participants trade contracts whose payoff depends on a future event, so that prices can be read as aggregate probability estimates. Prediction markets have forecast elections and project outcomes well, though in forecasting tournaments well-aggregated team forecasts have matched or beaten them. (Wolfers & Zitzewitz, 2004; Atanasov et al., 2017)

Artificial and collective intelligence
See also: Wisdom of crowds, Superforecasting
Read more: Prediction Markets as Intelligence Tools

R

Red teaming also: devil's advocacy

Assigning a group to take the perspective of an adversary or to challenge a plan, so that weaknesses and alternative outcomes are exposed before they are exploited. Devil's advocacy, in which one person argues against the prevailing view, is a lighter form of the same idea. (Zenko, 2015; Heuer, 1999)

Analysis and tradecraft
See also: Groupthink, Structured analytic techniques
Read more: Red Teaming and Devil's Advocacy

S

Scenario planning

A method for exploring several plausible futures, built from the key uncertainties and driving forces affecting an organisation, so that strategies can be tested against each. Its aim is to widen thinking and prepare for change rather than to predict a single outcome. (Schoemaker, 1995)

Analysis and tradecraft
See also: Weak signal, Indicators and warnings
Read more: Scenario Planning: Preparing for Several Futures

Source evaluation

Assessing information by the reliability of its source and the credibility of its content, often with a two-part grading such as the NATO Admiralty code (A–F for source reliability, 1–6 for information credibility). Research shows such ratings are often inconsistent and that analysts should consider a source's access and corroboration explicitly. (Irwin & Mandel, 2019)

Analysis and tradecraft
See also: Corroboration (Investigation), Open-source intelligence, Deception
Read more: Source Evaluation: Reliability, Access, and Corroboration

Structured analytic techniques also: SATs

Methods that make an analyst's reasoning explicit and open to review, such as analysis of competing hypotheses, key assumptions checks, red teaming, indicators and scenarios. They were promoted to counter the cognitive biases Heuer described, although evidence on how much they improve accuracy is mixed. (Heuer, 1999)

Analysis and tradecraft
See also: Analysis of competing hypotheses, Key assumptions check, Heuristics and biases (Insights)
Read more: Structured Analytic Techniques in Intelligence Analysis · Cognitive Bias Mitigation Training and Its Measurable Effects on Analytic Knowledge

Superforecasting

The practice of the top performers in the Good Judgment Project, a forecasting tournament run from 2011 to 2015, whose probability forecasts on geopolitical questions were markedly more accurate than those of other participants. Accuracy was linked to cognitive style, training in probabilistic reasoning, teamwork and frequent small updates. (Tetlock & Gardner, 2015; Mellers et al., 2015)

Analysis and tradecraft
See also: Brier score, Calibration, Wisdom of crowds
Read more: Superforecasting: What Research Says About Accurate Prediction

SWOT analysis

A planning framework that lists an organisation's internal strengths and weaknesses and its external opportunities and threats. It is easy to use and communicate but is often criticised for producing unprioritised lists; it is most useful when each item is backed by evidence and linked to decisions. (Porter, 1979)

Market and competitive intelligence
See also: Five forces, Competitive intelligence
Read more: SWOT and PESTLE Analysis: Uses and Limitations

W

Weak signal

An early, imprecise indication of a change that may become significant. Igor Ansoff argued in 1975 that firms should respond in graduated steps to weak signals of strategic surprise rather than wait for full information. Horizon scanning is the systematic search for such signals. (Ansoff, 1975)

Analysis and tradecraft
See also: Indicators and warnings, Scenario planning
Read more: Weak Signals and Horizon Scanning

Wisdom of crowds

The finding that the aggregate of many independent judgements is often more accurate than most individual judgements, as in Francis Galton's 1907 report that the median guess of an ox's weight at a fair was within 1% of the true value. It depends on diversity and independence of opinions; when judgements are correlated or influenced by each other the advantage shrinks. (Galton, 1907; Surowiecki, 2004)

Artificial and collective intelligence
See also: Collective intelligence, Prediction market
Read more: Collective Intelligence and the Wisdom of Crowds · When Crowds Are Wise and When They Are Not · Knowledge From Many Voices: The Epistemology of Aggregated Opinion

Words of estimative probability also: WEP

Verbal expressions such as "likely" or "almost certain" used in intelligence estimates to convey probability. Sherman Kent showed in 1964 that readers interpret such words very differently and proposed linking each to a numerical range. Many intelligence agencies now publish standard probability yardsticks. (Kent, 1964; Mandel & Barnes, 2014)

Analysis and tradecraft
See also: Calibration
Read more: Expressing Probability in Intelligence Estimates: Words of Estimative Probability

References

Ansoff, H. I. (1975). Managing strategic surprise by response to weak signals. California Management Review, 18(2), 21–33. https://doi.org/10.2307/41164635

Atanasov, P., Rescober, P., Stone, E., Swift, S. A., Servan-Schreiber, E., Tetlock, P., Ungar, L., & Mellers, B. (2017). Distilling the wisdom of crowds: Prediction markets vs. prediction polls. Management Science, 63(3), 691–706. https://doi.org/10.1287/mnsc.2015.2374

Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (pp. 610–623). ACM. https://doi.org/10.1145/3442188.3445922

Betts, R. K. (1978). Analysis, war, and decision: Why intelligence failures are inevitable. World Politics, 31(1), 61–89. https://doi.org/10.2307/2009967

Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1), 1–3. https://doi.org/10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2

Chen, H., Chiang, R. H. L., & Storey, V. C. (2012). Business intelligence and analytics: From big data to big impact. MIS Quarterly, 36(4), 1165–1188. https://doi.org/10.2307/41703503

Eppler, M. J., & Mengis, J. (2004). The concept of information overload: A review of literature from organization science, accounting, marketing, MIS, and related disciplines. The Information Society, 20(5), 325–344. https://doi.org/10.1080/01972240490507974

Galton, F. (1907). Vox populi. Nature, 75(1949), 450–451. https://doi.org/10.1038/075450a0

Glassman, M., & Kang, M. J. (2012). Intelligence in the internet age: The emergence and evolution of Open Source Intelligence (OSINT). Computers in Human Behavior, 28(2), 673–682. https://doi.org/10.1016/j.chb.2011.11.014

Heuer, R. J., Jr. (1999). Psychology of intelligence analysis. Center for the Study of Intelligence, Central Intelligence Agency.

Huber, G. P. (1991). Organizational learning: The contributing processes and the literatures. Organization Science, 2(1), 88–115. https://doi.org/10.1287/orsc.2.1.88

Hulnick, A. S. (2006). What's wrong with the intelligence cycle. Intelligence and National Security, 21(6), 959–979. https://doi.org/10.1080/02684520601046291

Irwin, D., & Mandel, D. R. (2019). Improving information evaluation for intelligence production. Intelligence and National Security, 34(4), 503–525. https://doi.org/10.1080/02684527.2019.1569343

Janis, I. L. (1972). Victims of groupthink: A psychological study of foreign-policy decisions and fiascoes. Houghton Mifflin.

Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12), Article 248. https://doi.org/10.1145/3571730

Kaplan, R. S., & Norton, D. P. (1992). The balanced scorecard: Measures that drive performance. Harvard Business Review, 70(1), 71–79.

Kent, S. (1949). Strategic intelligence for American world policy. Princeton University Press.

Kent, S. (1964). Words of estimative probability. Studies in Intelligence, 8(4), 49–65.

Luhn, H. P. (1958). A business intelligence system. IBM Journal of Research and Development, 2(4), 314–319. https://doi.org/10.1147/rd.24.0314

Mandel, D. R., & Barnes, A. (2014). Accuracy of forecasts in strategic intelligence. Proceedings of the National Academy of Sciences, 111(30), 10984–10989. https://doi.org/10.1073/pnas.1406138111

Mellers, B., Stone, E., Murray, T., Minster, A., Rohrbaugh, N., Bishop, M., Chen, E., Baker, J., Hou, Y., Horowitz, M., Ungar, L., & Tetlock, P. (2015). Identifying and cultivating superforecasters as a method of improving probabilistic predictions. Perspectives on Psychological Science, 10(3), 267–281. https://doi.org/10.1177/1745691615577794

Murphy, A. H. (1973). A new vector partition of the probability score. Journal of Applied Meteorology, 12(4), 595–600. https://doi.org/10.1175/1520-0450(1973)012<0595:ANVPOT>2.0.CO;2

Porter, M. E. (1979). How competitive forces shape strategy. Harvard Business Review, 57(2), 137–145.

Schoemaker, P. J. H. (1995). Scenario planning: A tool for strategic thinking. Sloan Management Review, 36(2), 25–40.

Surowiecki, J. (2004). The wisdom of crowds. Doubleday.

Tetlock, P. E., & Gardner, D. (2015). Superforecasting: The art and science of prediction. Crown.

Tufte, E. R. (1983). The visual display of quantitative information. Graphics Press.

Vosoughi, S., Roy, D., & Aral, S. (2018). The spread of true and false news online. Science, 359(6380), 1146–1151. https://doi.org/10.1126/science.aap9559

Wardle, C., & Derakhshan, H. (2017). Information disorder: Toward an interdisciplinary framework for research and policy making (Council of Europe report DGI(2017)09). Council of Europe.

Wilensky, H. L. (1967). Organizational intelligence: Knowledge and policy in government and industry. Basic Books.

Wolfers, J., & Zitzewitz, E. (2004). Prediction markets. Journal of Economic Perspectives, 18(2), 107–126. https://doi.org/10.1257/0895330041371321

Woolley, A. W., Chabris, C. F., Pentland, A., Hashmi, N., & Malone, T. W. (2010). Evidence for a collective intelligence factor in the performance of human groups. Science, 330(6004), 686–688. https://doi.org/10.1126/science.1193147

Zenko, M. (2015). Red team: How to succeed by thinking like the enemy. Basic Books.