Skip to content

Statistics

Statistics glossary

Terms for describing data, drawing inferences, building models and reasoning about cause, with the common ways statistics mislead.

B

Base rate neglect also: base rate fallacy

The tendency to ignore the prior frequency of an event when judging the probability of a case, relying instead on specific information such as a test result. It leads people to overestimate the chance that a positive test means disease when the disease is rare. Presenting information as natural frequencies (for example, 9 out of 1,000) greatly reduces the error. (Bar-Hillel, 1980; Gigerenzer & Hoffrage, 1995)

Communication and misuse
See also: Bayes' theorem, Prior probability
Read more: Base Rate Neglect and the Misreading of Probabilities · Bayesian Reasoning in Everyday Inquiry

Bayes' theorem

A rule for updating the probability of a hypothesis in light of evidence: the posterior probability is proportional to the prior probability times the likelihood of the evidence under the hypothesis. Formally, P(H | E) = P(E | H) × P(H) / P(E). It underlies Bayesian statistics and much of rational belief revision. (Bayes, 1763; Jaynes, 2003)

Bayesian and computational
See also: Prior probability, Bayesian inference, Base rate neglect
Read more: Bayes' Theorem and Updating Beliefs With Evidence · Bayesian Reasoning in Everyday Inquiry

Bayesian inference

An approach to statistics in which unknown quantities are treated as having probability distributions. A prior distribution expressing what is known before the data is combined with the likelihood of the data to give a posterior distribution, from which estimates and credible intervals are read directly as probability statements about the parameters. (Edwards et al., 1963; Jaynes, 2003)

Bayesian and computational
See also: Bayes' theorem, Frequentist inference, Prior probability
Read more: Frequentist and Bayesian Statistics: Two Philosophies of Probability · Bayesian Inference in Market Research

Bootstrap

A resampling method introduced by Bradley Efron in 1979 that estimates the sampling distribution of a statistic by repeatedly drawing samples, with replacement, from the observed data and recomputing the statistic each time. The spread of the resampled values gives standard errors and confidence intervals without strong distributional assumptions. (Efron, 1979)

Bayesian and computational
See also: Standard error, Confidence interval, Monte Carlo simulation
Read more: Bootstrap Methods and Resampling

C

Central limit theorem

The result that the distribution of the mean (or sum) of many independent observations with finite variance approaches a normal distribution as the sample size grows, whatever the shape of the original distribution. It explains why the normal distribution is used so widely for sampling distributions and confidence intervals. (Lohr, 2021)

Foundations
See also: Normal distribution, Sampling distribution, Law of large numbers
Read more: The Central Limit Theorem Explained

Chi-square test

A family of tests introduced by Karl Pearson in 1900 that compare observed frequencies with those expected under a hypothesis. The test of independence asks whether two categorical variables in a cross-tabulation are associated. With complex survey samples, design-adjusted versions such as the Rao–Scott correction are needed. (Pearson, 1900)

Tests and models
See also: P-value
Read more: Chi-Square Tests for Categorical Survey Data · Significance Testing in Cross-Tabulated Survey Results

Cluster analysis

Methods that group cases so that cases in the same cluster are more similar to each other than to those in other clusters, using distances such as Euclidean distance. K-means and hierarchical clustering are common. In market research it is used to build segments, whose usefulness depends on the variables chosen. (Hastie et al., 2009)

Tests and models
See also: Segmentation (Research), Latent class analysis
Read more: Cluster Analysis for Market Segmentation

Cohen's kappa

A measure of agreement between two raters assigning cases to categories, corrected for the agreement expected by chance. Kappa is 1 for perfect agreement and 0 for chance-level agreement. It is widely used to check the reliability of coding in content analysis and qualitative research. (Cohen, 1960)

Survey statistics
See also: Inter-rater reliability, Reliability (Research)
Read more: Inter-Rater Reliability: Cohen's Kappa and Related Measures

Collider bias

Bias that arises when an analysis conditions on (selects on, stratifies by or adjusts for) a variable that is caused by both the exposure and the outcome. Conditioning on such a collider creates a spurious association between its causes, as in Berkson's finding that two diseases appear related in hospital samples. (Elwert & Winship, 2014; Berkson, 1946; Hernán et al., 2004)

Causal inference
See also: Confounding, Directed acyclic graph, Selection bias
Read more: Correlation Is Not Causation: Spurious Relationships in Data · Directed Acyclic Graphs for Causal Reasoning

Confidence interval

A range computed from sample data by a procedure that, over repeated samples, contains the true parameter a stated proportion of the time, such as 95%. A particular interval either contains the parameter or does not; the 95% describes the procedure, not the probability for one interval. Jerzy Neyman set out the theory in 1937. (Neyman, 1937; Greenland et al., 2016)

Inference
See also: Margin of error (Surveys), Standard error, P-value
Read more: Confidence Intervals: Meaning and Misinterpretation

Confounding

Distortion of the estimated effect of an exposure on an outcome because a third variable causes both. Confounding can create, hide or reverse an association. It is controlled by randomisation, or in observational data by adjustment, matching or stratification on the confounders, which requires that they are known and measured. (Pearl, 2009; Hernán et al., 2004)

Causal inference
See also: Collider bias, Directed acyclic graph, Simpson's paradox, Spurious correlation
Read more: Confounding Variables and How to Control for Them · Correlation Is Not Causation: Spurious Relationships in Data

Correlation coefficient

A number between −1 and +1 summarising the strength and direction of association between two variables. Pearson's r measures linear association; Spearman's rho measures monotonic association using ranks. A correlation does not show causation and can be distorted by outliers and by non-linear relationships, as Anscombe's quartet illustrates. (Anscombe, 1973)

Tests and models
See also: Spurious correlation, Confounding, Regression
Read more: Correlation: Pearson, Spearman and Their Limits · Correlation Is Not Causation: Spurious Relationships in Data

Counterfactual

The outcome that would have occurred under a condition other than the one that actually happened. In the potential outcomes framework each unit has an outcome under treatment and under control, and the causal effect is their difference. Because only one is ever observed, causal inference is a problem of estimating the missing counterfactual. (Rubin, 1974; Holland, 1986)

Causal inference
See also: Potential outcomes, Randomised controlled trial (Research)
Read more: Causal Inference: From Association to Cause · Using Counterfactual Thinking to Improve Interpretation

Cronbach's alpha

A coefficient of internal consistency for a multi-item scale, based on the number of items and the average inter-item covariance. Values closer to 1 indicate that items vary together. A high alpha does not prove that a scale measures a single construct, and alpha rises with the number of items. (Cronbach, 1951)

Survey statistics
See also: Reliability (Research), Likert scale (Surveys), Factor analysis
Read more: Reliability Statistics: Cronbach's Alpha and Alternatives

Cross-validation

A method for estimating how well a model will predict new data by repeatedly fitting it on part of the data and testing it on the held-out remainder. In k-fold cross-validation the data are split into k parts and each part serves once as the test set. It is used to choose model complexity and avoid overfitting. (Hastie et al., 2009)

Data science and machine learning
See also: Overfitting, Model selection criteria
Read more: Predictive Modelling: Training, Testing and Validation · Model Selection Criteria (AIC, BIC, Cross-Validation) and Their Epistemic Implications

D

Difference-in-differences also: DiD

A method that estimates a causal effect by comparing the change in an outcome over time in a group exposed to a policy or event with the change in an unexposed group. It removes stable differences between the groups and common trends, under the key assumption that the groups would have followed parallel trends without the intervention. (Card & Krueger, 1994; Angrist & Pischke, 2009)

Causal inference
See also: Quasi-experiment (Research), Synthetic control method, Counterfactual
Read more: Difference-in-Differences Analysis

Differential privacy

A mathematical definition of privacy for statistical releases: an algorithm is differentially private if its output distribution changes very little whether or not any one person's data are included. It is usually achieved by adding calibrated random noise, and it gives a measurable privacy budget, epsilon. The US Census Bureau used it for 2020 census products. (Dwork et al., 2006)

Communication and misuse
Read more: Data Privacy in Statistics: Disclosure Control and Differential Privacy

Directed acyclic graph also: DAG

A diagram of variables (nodes) connected by arrows representing direct causal effects, with no path that returns to its starting point. DAGs make causal assumptions explicit and show which variables must be adjusted for to estimate an effect (the back-door criterion) and which must not, such as colliders and mediators. (Pearl, 2009; Pearl & Mackenzie, 2018)

Causal inference
See also: Confounding, Collider bias
Read more: Directed Acyclic Graphs for Causal Reasoning

E

Ecological fallacy

Inferring relationships about individuals from data on groups. William Robinson showed in 1950 that the correlation between literacy and foreign birth across US states had the opposite sign to the correlation among individuals. Group-level correlations can be strong even when individual-level ones are weak or reversed. (Robinson, 1950)

Communication and misuse
See also: Simpson's paradox, Spurious correlation
Read more: Simpson's Paradox: When Aggregated Data Mislead · Correlation Is Not Causation: Spurious Relationships in Data

Effect size

A quantitative measure of the magnitude of a difference or relationship, such as a mean difference, Cohen's d, an odds ratio or a correlation. Unlike a p-value it does not depend on sample size, so it indicates practical importance. Jacob Cohen's conventions of small, medium and large effects are widely used but were meant only as a fallback. (Cohen, 1988)

Inference
See also: Statistical power, P-value, Meta-analysis (Research)
Read more: Effect Sizes: Reporting What Matters Beyond Significance · Statistical Significance Versus Practical Significance

F

Factor analysis

A family of methods that explain the correlations among many observed variables by a smaller number of unobserved (latent) factors. Exploratory factor analysis looks for structure; confirmatory factor analysis tests a structure specified in advance. Charles Spearman introduced the approach in 1904 in his work on general intelligence. (Spearman, 1904)

Tests and models
See also: Principal component analysis, Measurement invariance (Surveys), Cronbach's alpha
Read more: Factor Analysis: Exploratory and Confirmatory

False discovery rate also: FDR

The expected proportion of rejected null hypotheses that are actually true, that is, the share of "discoveries" that are false. The Benjamini–Hochberg procedure controls it when many tests are performed, offering more power than methods that control the chance of any false positive. (Benjamini & Hochberg, 1995)

Inference
See also: Multiple comparisons, Type I error
Read more: Multiple Comparisons and the Problem of False Discoveries

Frequentist inference

An approach to statistics in which probability means long-run relative frequency, parameters are fixed but unknown, and methods are judged by how they would perform over repeated samples. Confidence intervals, p-values and hypothesis tests are frequentist tools. (Neyman, 1937)

Foundations
See also: Bayesian inference, Confidence interval, P-value
Read more: Frequentist and Bayesian Statistics: Two Philosophies of Probability

G

Generalised linear model also: GLM

An extension of linear regression, introduced by Nelder and Wedderburn in 1972, in which the outcome can follow any distribution from the exponential family (normal, binomial, Poisson and others) and is related to the predictors through a link function. Logistic and Poisson regression are special cases. (Nelder & Wedderburn, 1972)

Tests and models
See also: Logistic regression, Regression
Read more: Generalized Linear Models and Their Extensions for Diverse Outcome Types · Logistic Regression for Binary Outcomes

I

Instrumental variable also: IV

A variable that affects the treatment, is unrelated to unmeasured confounders, and affects the outcome only through the treatment. Under these assumptions it can be used to estimate a causal effect from observational data, typically the effect for "compliers" whose treatment is changed by the instrument. (Angrist et al., 1996)

Causal inference
See also: Confounding, Natural experiment (Research)
Read more: Instrumental Variables

Inter-rater reliability

The degree to which different coders or raters give consistent judgements of the same material. Simple percentage agreement overstates it because some agreement occurs by chance; chance-corrected statistics such as Cohen's kappa or Krippendorff's alpha are preferred. (Cohen, 1960; Krippendorff, 2019)

Survey statistics
See also: Cohen's kappa, Reliability (Research), Content analysis (Research)
Read more: Inter-Rater Reliability: Cohen's Kappa and Related Measures

L

Lasso

Least absolute shrinkage and selection operator: a regression method that adds a penalty proportional to the sum of the absolute values of the coefficients. The penalty shrinks coefficients towards zero and sets some exactly to zero, so it performs variable selection and reduces overfitting when there are many predictors. (Tibshirani, 1996; Hastie et al., 2009)

Data science and machine learning
See also: Overfitting, Regression
Read more: Penalized Regression (Lasso, Ridge, Elastic Net) for High-Dimensional Knowledge

Latent class analysis

A statistical model that assumes the population consists of a number of unobserved classes and estimates, from categorical survey answers, the probability of each answer in each class and each respondent's probability of class membership. It is a model-based alternative to cluster analysis for segmentation. (Hastie et al., 2009)

Tests and models
See also: Cluster analysis, Segmentation (Research)
Read more: Latent Class Analysis

Law of large numbers

The theorem that the average of independent observations from the same distribution converges to the expected value as the number of observations grows. It explains why large random samples give stable estimates, and it does not imply that short runs "even out", the gambler's fallacy. (Jaynes, 2003)

Foundations
See also: Central limit theorem, Sampling distribution
Read more: The Law of Large Numbers and Its Practical Meaning

Levels of measurement also: scales of measurement

Stanley Stevens's classification of variables as nominal (categories), ordinal (ordered categories), interval (equal differences, arbitrary zero) or ratio (equal differences, true zero). The level determines which statistics are meaningful, although applying the rules strictly, for example to rating scales, remains debated. (Stevens, 1946)

Foundations
See also: Likert scale (Surveys)
Read more: Levels of Measurement: Nominal, Ordinal, Interval and Ratio

Logistic regression

A regression model for a binary outcome in which the log-odds of the outcome is a linear function of the predictors. Exponentiated coefficients are odds ratios. It is a generalised linear model with a binomial distribution and a logit link. (Nelder & Wedderburn, 1972)

Tests and models
See also: Generalised linear model, Odds ratio
Read more: Logistic Regression for Binary Outcomes

M

Model selection criteria also: AIC, BIC

Measures for choosing among statistical models that balance fit against complexity. The Akaike information criterion (AIC) estimates out-of-sample prediction error; the Bayesian information criterion (BIC) penalises extra parameters more heavily as sample size grows. Lower values indicate a preferred model. (Akaike, 1974)

Data science and machine learning
See also: Cross-validation, Overfitting
Read more: Model Selection Criteria (AIC, BIC, Cross-Validation) and Their Epistemic Implications

Monte Carlo simulation

Using repeated random sampling to approximate quantities that are hard to calculate exactly, such as the distribution of outcomes of a risky project or the behaviour of a statistical method. Metropolis and Ulam named and described the method in 1949, drawing on work at Los Alamos. (Metropolis & Ulam, 1949)

Bayesian and computational
See also: Bootstrap
Read more: Monte Carlo Simulation for Risk and Uncertainty · Simulation Studies for Evaluating Statistical Method Performance

MRP also: multilevel regression and post-stratification

Multilevel regression and post-stratification: a method that fits a multilevel model predicting a survey outcome from demographic and geographic variables, then weights the predictions for each population cell by its known size from census data. It produces estimates for small groups and areas and can correct for unrepresentative samples. (Park et al., 2004)

Bayesian and computational
See also: Multilevel model, Weighting (Surveys), Small area estimation
Read more: Multilevel Regression and Post-Stratification (MRP)

Multilevel model also: hierarchical model

A regression model for data with a nested structure, such as employees within firms or repeated measures within people, that includes random effects at each level. It estimates variation within and between groups and "partially pools" estimates for small groups towards the overall mean. (Gelman & Hill, 2007)

Tests and models
See also: MRP, Small area estimation
Read more: Multilevel and Hierarchical Models · Bayesian Hierarchical Models for Partial Pooling and Knowledge Stabilization

Multiple comparisons

The problem that when many hypothesis tests are performed, the chance that at least one gives a false positive grows well above the nominal significance level. Corrections such as Bonferroni (controlling the family-wise error rate) and Benjamini–Hochberg (controlling the false discovery rate) adjust for it. (Benjamini & Hochberg, 1995)

Inference
See also: False discovery rate, P-hacking (Research), Type I error
Read more: Multiple Comparisons and the Problem of False Discoveries

N

Normal distribution also: Gaussian distribution

The symmetric, bell-shaped continuous distribution defined by its mean and standard deviation, with about 68% of values within one standard deviation of the mean and about 95% within two. Its importance comes largely from the central limit theorem. (Lohr, 2021)

Distributions
See also: Central limit theorem, Standard deviation
Read more: The Normal Distribution and Why It Appears Everywhere

O

Odds ratio

The ratio of the odds of an outcome in one group to the odds in another; a value of 1 means no association. It is the natural effect measure in logistic regression and case–control studies. When the outcome is common, an odds ratio exaggerates the corresponding risk ratio. (Nelder & Wedderburn, 1972)

Tests and models
See also: Logistic regression, Effect size
Read more: Logistic Regression for Binary Outcomes

Overfitting

Fitting a model so closely to the particular sample that it captures noise as well as signal, so that it performs well on the training data but poorly on new data. It is the high-variance side of the bias–variance trade-off and is controlled by cross-validation, regularisation and simpler models. (Hastie et al., 2009)

Data science and machine learning
See also: Lasso, Cross-validation
Read more: Overfitting and the Bias-Variance Trade-off · Predictive Modelling: Training, Testing and Validation

P

Potential outcomes also: Rubin causal model

A framework for causal inference, associated with Donald Rubin, in which each unit has a potential outcome under each possible treatment. The causal effect is the difference between them, and since only one is observed, the "fundamental problem of causal inference", effects are estimated by comparing groups under assumptions such as random assignment. (Rubin, 1974; Holland, 1986)

Causal inference
See also: Counterfactual, Randomised controlled trial (Research)
Read more: Causal Inference: From Association to Cause

Principal component analysis also: PCA

A dimension-reduction method, first described by Karl Pearson in 1901, that transforms correlated variables into a smaller set of uncorrelated components that capture as much of the total variance as possible. Unlike factor analysis it does not model a latent cause of the variables. (Pearson, 1901)

Tests and models
See also: Factor analysis
Read more: Principal Component Analysis and Dimension Reduction

Prior probability also: prior

In Bayesian reasoning, the probability assigned to a hypothesis before the current evidence is considered, often based on base rates or earlier studies. Combined with the likelihood of the evidence, it gives the posterior probability. Neglecting the prior is the source of base rate neglect. (Bayes, 1763; Jaynes, 2003)

Bayesian and computational
See also: Bayes' theorem, Base rate neglect, Bayesian inference
Read more: Bayes' Theorem and Updating Beliefs With Evidence · Bayesian Reasoning in Everyday Inquiry

Propensity score

The probability that a unit receives a treatment given its observed characteristics. Rosenbaum and Rubin showed that matching, weighting or stratifying on this single score can balance the observed covariates between treated and untreated groups. It does not correct for unmeasured confounders. (Rosenbaum & Rubin, 1983)

Causal inference
See also: Confounding, Weighting (Surveys)
Read more: Propensity Score Matching · Propensity Score Adjustment for Online Survey Samples

P-value

The probability, computed assuming the null hypothesis and all model assumptions are true, of obtaining a result at least as extreme as the one observed. It is not the probability that the null hypothesis is true, nor a measure of effect size. The American Statistical Association's 2016 statement warns against basing conclusions on whether p crosses 0.05. (Wasserstein & Lazar, 2016; Greenland et al., 2016)

Inference
See also: Statistical significance, Confidence interval, Effect size, P-hacking (Research)
Read more: P-Values: What They Mean and What They Do Not · The ASA Statement on P-Values and the Move Beyond Significance

R

Regression

A family of methods that model how the expected value of an outcome changes with one or more predictors. In linear regression, coefficients estimate the change in the outcome per unit change in a predictor, holding the other predictors constant. Galton coined the term from his observation of regression to the mean in heredity. (Galton, 1886; Gelman & Hill, 2007)

Tests and models
See also: Logistic regression, Generalised linear model, Confounding
Read more: Simple and Multiple Linear Regression

Regression discontinuity design also: RDD

A quasi-experimental design used when treatment is assigned by whether a score crosses a cut-off, such as a test mark for a scholarship. Comparing units just above and just below the threshold estimates the causal effect at the cut-off, because those units are otherwise similar. (Thistlethwaite & Campbell, 1960; Angrist & Pischke, 2009)

Causal inference
See also: Quasi-experiment (Research), Instrumental variable
Read more: Regression Discontinuity Designs

Regression to the mean

The tendency for unusually high or low measurements to be followed by values closer to the average, because extreme values partly reflect chance. Francis Galton described it in heredity. Ignoring it leads people to credit interventions, or blame them, for changes that would have happened anyway. (Galton, 1886; Kahneman & Tversky, 1973)

Communication and misuse
See also: Selection bias
Read more: Regression to the Mean in Business Performance

S

Sampling distribution

The distribution of a statistic, such as a sample mean, across all possible samples of the same size from a population. Its standard deviation is the standard error. It is the basis of confidence intervals and hypothesis tests. (Lohr, 2021)

Distributions
See also: Standard error, Central limit theorem, Bootstrap
Read more: Sampling Distributions and Standard Errors

Selection bias

Bias that arises when the units included in an analysis differ systematically from the population of interest because of how they were selected, or because selection depends on both exposure and outcome. Coverage error, nonresponse bias, survivorship bias and collider bias are forms of it. (Hernán et al., 2004)

Causal inference
See also: Collider bias, Survivorship bias, Nonresponse bias (Surveys)
Read more: Survivorship Bias in Data and Decisions · Correlation Is Not Causation: Spurious Relationships in Data

Simpson's paradox

A reversal in which an association between two variables in every subgroup disappears or reverses when the subgroups are combined. The Berkeley graduate admissions data are a classic example. Which comparison is correct depends on the causal structure, not on the arithmetic. (Simpson, 1951; Bickel et al., 1975; Pearl, 2009)

Communication and misuse
See also: Confounding, Ecological fallacy
Read more: Simpson's Paradox: When Aggregated Data Mislead · Correlation Is Not Causation: Spurious Relationships in Data

Small area estimation

Methods for producing reliable estimates for geographic areas or groups where the survey sample is too small for direct estimates. Models borrow strength from other areas and from auxiliary data such as census or administrative records. (Rao & Molina, 2015)

Survey statistics
See also: MRP, Multilevel model
Read more: Small Area Estimation

Spurious correlation

An association between two variables that does not reflect a causal link between them, because both are driven by a third factor, because of chance, because of shared trends over time, or because of how the variables were constructed. Karl Pearson used the term in 1897 for correlations between ratios with a common denominator. (Pearson, 1897; Yule, 1926; Messerli, 2012)

Communication and misuse
See also: Confounding, Correlation coefficient
Read more: Correlation Is Not Causation: Spurious Relationships in Data

Standard deviation

A measure of spread equal to the square root of the variance, the average squared distance of values from their mean. It is expressed in the units of the data. For roughly normal data, about two-thirds of values lie within one standard deviation of the mean. (Tukey, 1977)

Descriptive statistics
See also: Normal distribution, Standard error
Read more: Measures of Dispersion: Range, Variance and Standard Deviation

Standard error

The standard deviation of the sampling distribution of a statistic, measuring how much the statistic would vary from sample to sample. For a mean from a simple random sample it equals the standard deviation divided by the square root of the sample size. Complex designs require design-based variance estimation. (Lohr, 2021; Kish, 1965)

Distributions
See also: Sampling distribution, Confidence interval, Design effect (Surveys)
Read more: Sampling Distributions and Standard Errors · Variance Estimation in Complex Survey Samples

Statistical power

The probability that a test will detect an effect of a given size if it exists, that is, one minus the probability of a Type II error. Power depends on the effect size, the sample size, the significance level and the variability of the data. Underpowered studies miss real effects and exaggerate the effects they do find. (Cohen, 1988)

Inference
See also: Type I error, Effect size, Type I error
Read more: Type I and Type II Errors and Statistical Power · Power Analysis for Planning Studies

Statistical process control also: SPC

Monitoring a process over time with control charts that separate common-cause variation, inherent in a stable process, from special-cause variation that signals a change worth investigating. Walter Shewhart developed the method at Bell Laboratories in the 1920s. (Shewhart, 1931)

Communication and misuse
Read more: Statistical Process Control and Monitoring for Ongoing Knowledge Quality

Statistical significance

The label applied when a p-value falls below a chosen threshold, conventionally 0.05. A significant result is not necessarily important, and a non-significant result is not evidence of no effect. Many statisticians recommend reporting effect sizes and confidence intervals instead of dichotomising results. (Wasserstein & Lazar, 2016; Greenland et al., 2016)

Inference
See also: P-value, Effect size, Type I error
Read more: Statistical Significance Versus Practical Significance · The ASA Statement on P-Values and the Move Beyond Significance

Survivorship bias

Drawing conclusions only from cases that passed a selection process, such as successful firms or returning aircraft, while overlooking those that did not. Abraham Wald's Second World War analysis of where to armour bombers, based on where surviving planes were not hit, is the classic example. (Mangel & Samaniego, 1984)

Communication and misuse
See also: Selection bias, Collider bias
Read more: Survivorship Bias in Data and Decisions

Synthetic control method

A method for estimating the effect of an intervention on a single unit, such as a region, by constructing a weighted combination of untreated units that closely matches the treated unit before the intervention. The divergence between the treated unit and its synthetic control afterwards estimates the effect. (Abadie et al., 2010)

Causal inference
See also: Difference-in-differences, Counterfactual
Read more: Synthetic Control Methods for Comparative Case Knowledge

T

t-test also: Student's t-test

A test comparing a mean with a value, or two means with each other, using the t distribution, which accounts for the extra uncertainty from estimating the standard deviation in small samples. William Sealy Gosset published it in 1908 under the pseudonym "Student". (Student, 1908)

Tests and models
See also: P-value, Effect size
Read more: The t-Test: Comparing Two Means

Type I error

Rejecting a null hypothesis that is true, a false positive. Its probability is set by the significance level, often 0.05. A Type II error is failing to reject a false null hypothesis, a false negative; its probability is one minus the power. (Cohen, 1988)

Inference
See also: Statistical power, Multiple comparisons, False discovery rate
Read more: Type I and Type II Errors and Statistical Power

W

Weighted mean

An average in which each value counts in proportion to a weight, calculated as the sum of each value times its weight divided by the sum of the weights. In surveys the weights represent how many population members each respondent stands for, so the weighted mean estimates the population mean. (Horvitz & Thompson, 1952; Kish, 1992)

Descriptive statistics
See also: Weighting (Surveys), Effective sample size (Surveys)
Read more: Weighted Means and Weighted Percentages · Statistical Analysis of Weighted Survey Data

References

Abadie, A., Diamond, A., & Hainmueller, J. (2010). Synthetic control methods for comparative case studies: Estimating the effect of California's tobacco control program. Journal of the American Statistical Association, 105(490), 493–505. https://doi.org/10.1198/jasa.2009.ap08746

Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716–723. https://doi.org/10.1109/TAC.1974.1100705

Angrist, J. D., & Pischke, J.-S. (2009). Mostly harmless econometrics: An empiricist's companion. Princeton University Press. https://doi.org/10.1515/9781400829828

Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91(434), 444–455. https://doi.org/10.1080/01621459.1996.10476902

Anscombe, F. J. (1973). Graphs in statistical analysis. The American Statistician, 27(1), 17–21. https://doi.org/10.1080/00031305.1973.10478966

Bar-Hillel, M. (1980). The base-rate fallacy in probability judgments. Acta Psychologica, 44(3), 211–233. https://doi.org/10.1016/0001-6918(80)90046-3

Bayes, T. (1763). An essay towards solving a problem in the doctrine of chances. Philosophical Transactions of the Royal Society of London, 53, 370–418. https://doi.org/10.1098/rstl.1763.0053

Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57(1), 289–300. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x

Berkson, J. (1946). Limitations of the application of fourfold table analysis to hospital data. Biometrics Bulletin, 2(3), 47–53. https://doi.org/10.2307/3002000

Bickel, P. J., Hammel, E. A., & O'Connell, J. W. (1975). Sex bias in graduate admissions: Data from Berkeley. Science, 187(4175), 398–404. https://doi.org/10.1126/science.187.4175.398

Card, D., & Krueger, A. B. (1994). Minimum wages and employment: A case study of the fast-food industry in New Jersey and Pennsylvania. American Economic Review, 84(4), 772–793.

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46. https://doi.org/10.1177/001316446002000104

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/BF02310555

Dwork, C., McSherry, F., Nissim, K., & Smith, A. (2006). Calibrating noise to sensitivity in private data analysis. In S. Halevi & T. Rabin (Eds.), Theory of cryptography (pp. 265–284). Springer. https://doi.org/10.1007/11681878_14

Edwards, W., Lindman, H., & Savage, L. J. (1963). Bayesian statistical inference for psychological research. Psychological Review, 70(3), 193–242. https://doi.org/10.1037/h0044139

Efron, B. (1979). Bootstrap methods: Another look at the jackknife. The Annals of Statistics, 7(1), 1–26. https://doi.org/10.1214/aos/1176344552

Elwert, F., & Winship, C. (2014). Endogenous selection bias: The problem of conditioning on a collider variable. Annual Review of Sociology, 40, 31–53. https://doi.org/10.1146/annurev-soc-071913-043455

Galton, F. (1886). Regression towards mediocrity in hereditary stature. Journal of the Anthropological Institute of Great Britain and Ireland, 15, 246–263. https://doi.org/10.2307/2841583

Gelman, A., & Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. Cambridge University Press. https://doi.org/10.1017/CBO9780511790942

Gigerenzer, G., & Hoffrage, U. (1995). How to improve Bayesian reasoning without instruction: Frequency formats. Psychological Review, 102(4), 684–704. https://doi.org/10.1037/0033-295X.102.4.684

Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016). Statistical tests, P values, confidence intervals, and power: A guide to misinterpretations. European Journal of Epidemiology, 31(4), 337–350. https://doi.org/10.1007/s10654-016-0149-3

Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning: Data mining, inference, and prediction (2nd ed.). Springer. https://doi.org/10.1007/978-0-387-84858-7

Hernán, M. A., Hernández-Díaz, S., & Robins, J. M. (2004). A structural approach to selection bias. Epidemiology, 15(5), 615–625. https://doi.org/10.1097/01.ede.0000135174.63482.43

Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81(396), 945–960. https://doi.org/10.1080/01621459.1986.10478354

Horvitz, D. G., & Thompson, D. J. (1952). A generalization of sampling without replacement from a finite universe. Journal of the American Statistical Association, 47(260), 663–685. https://doi.org/10.1080/01621459.1952.10483446

Jaynes, E. T. (2003). Probability theory: The logic of science (G. L. Bretthorst, Ed.). Cambridge University Press. https://doi.org/10.1017/CBO9780511790423

Kahneman, D., & Tversky, A. (1973). On the psychology of prediction. Psychological Review, 80(4), 237–251. https://doi.org/10.1037/h0034747

Kish, L. (1965). Survey sampling. Wiley.

Kish, L. (1992). Weighting for unequal Pi. Journal of Official Statistics, 8(2), 183–200.

Krippendorff, K. (2019). Content analysis: An introduction to its methodology (4th ed.). SAGE.

Lohr, S. L. (2021). Sampling: Design and analysis (3rd ed.). CRC Press. https://doi.org/10.1201/9780429298899

Mangel, M., & Samaniego, F. J. (1984). Abraham Wald's work on aircraft survivability. Journal of the American Statistical Association, 79(386), 259–267. https://doi.org/10.1080/01621459.1984.10478038

Messerli, F. H. (2012). Chocolate consumption, cognitive function, and Nobel laureates. New England Journal of Medicine, 367(16), 1562–1564. https://doi.org/10.1056/NEJMon1211064

Metropolis, N., & Ulam, S. (1949). The Monte Carlo method. Journal of the American Statistical Association, 44(247), 335–341. https://doi.org/10.1080/01621459.1949.10483310

Nelder, J. A., & Wedderburn, R. W. M. (1972). Generalized linear models. Journal of the Royal Statistical Society: Series A (General), 135(3), 370–384. https://doi.org/10.2307/2344614

Neyman, J. (1937). Outline of a theory of statistical estimation based on the classical theory of probability. Philosophical Transactions of the Royal Society of London. Series A, 236(767), 333–380. https://doi.org/10.1098/rsta.1937.0005

Park, D. K., Gelman, A., & Bafumi, J. (2004). Bayesian multilevel estimation with poststratification: State-level estimates from national polls. Political Analysis, 12(4), 375–385. https://doi.org/10.1093/pan/mph024

Pearl, J., & Mackenzie, D. (2018). The book of why: The new science of cause and effect. Basic Books.

Pearl, J. (2009). Causality: Models, reasoning, and inference (2nd ed.). Cambridge University Press. https://doi.org/10.1017/CBO9780511803161

Pearson, K. (1897). Mathematical contributions to the theory of evolution: On a form of spurious correlation which may arise when indices are used in the measurement of organs. Proceedings of the Royal Society of London, 60, 489–498. https://doi.org/10.1098/rspl.1896.0076

Pearson, K. (1900). On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 50(302), 157–175. https://doi.org/10.1080/14786440009463897

Pearson, K. (1901). On lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 2(11), 559–572. https://doi.org/10.1080/14786440109462720

Rao, J. N. K., & Molina, I. (2015). Small area estimation (2nd ed.). Wiley. https://doi.org/10.1002/9781118735855

Robinson, W. S. (1950). Ecological correlations and the behavior of individuals. American Sociological Review, 15(3), 351–357. https://doi.org/10.2307/2087176

Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41–55. https://doi.org/10.1093/biomet/70.1.41

Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5), 688–701. https://doi.org/10.1037/h0037350

Shewhart, W. A. (1931). Economic control of quality of manufactured product. Van Nostrand.

Simpson, E. H. (1951). The interpretation of interaction in contingency tables. Journal of the Royal Statistical Society: Series B (Methodological), 13(2), 238–241. https://doi.org/10.1111/j.2517-6161.1951.tb00088.x

Spearman, C. (1904). "General intelligence," objectively determined and measured. The American Journal of Psychology, 15(2), 201–292. https://doi.org/10.2307/1412107

Stevens, S. S. (1946). On the theory of scales of measurement. Science, 103(2684), 677–680. https://doi.org/10.1126/science.103.2684.677

Student. (1908). The probable error of a mean. Biometrika, 6(1), 1–25. https://doi.org/10.1093/biomet/6.1.1

Thistlethwaite, D. L., & Campbell, D. T. (1960). Regression-discontinuity analysis: An alternative to the ex post facto experiment. Journal of Educational Psychology, 51(6), 309–317. https://doi.org/10.1037/h0044319

Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1), 267–288. https://doi.org/10.1111/j.2517-6161.1996.tb02080.x

Tukey, J. W. (1977). Exploratory data analysis. Addison-Wesley.

Wasserstein, R. L., & Lazar, N. A. (2016). The ASA statement on p-values: Context, process, and purpose. The American Statistician, 70(2), 129–133. https://doi.org/10.1080/00031305.2016.1154108

Yule, G. U. (1926). Why do we sometimes get nonsense-correlations between time-series? A study in sampling and the nature of time-series. Journal of the Royal Statistical Society, 89(1), 1–63. https://doi.org/10.2307/2341482