Discipline 5 of 7
Statistics
The numbers behind the findings: describing data, testing ideas and reading results correctly.
Statistics turns individual answers into reliable numbers: percentages, averages, margins of error and tests that show whether a difference is real or a product of chance.
These articles explain how data is described, how conclusions are drawn from samples, and how to read statistical results without being misled.
Foundations9 articles
- What Statistics Is and Why It Matters for Decision-Making
What statistics is, how it turns data into evidence under uncertainty, and why sound statistical thinking leads to better business and survey decisions than intuition or raw data alone.
8 min read - A Short History of Statistics: From Political Arithmetic to Data Science
How statistics grew from counting births and deaths for the state into a science of uncertainty, and why that history still shapes how surveys and business data are analysed today.
7 min read - Populations, Samples, Parameters and Statistics
The four ideas behind every survey estimate: what a population and a sample are, how a parameter differs from a statistic, and why sample quality matters more than sample size.
13 min read - Levels of Measurement: Nominal, Ordinal, Interval and Ratio
What the four levels of measurement mean, which summaries and tests suit each one, and how to handle rating scales and other awkward survey variables without misleading anyone.
10 min read - Variables and Data Types in Applied Statistics
What variables are, how nominal, ordinal and numeric data differ, why the type of a variable shapes every later analysis, and how to code survey and business data so it stays useful.
12 min read - Frequentist and Bayesian Statistics: Two Philosophies of Probability
What probability means to frequentist and Bayesian statisticians, how each school turns survey data into conclusions, a worked example on purchase intention, and when each approach serves business decisions best.
8 min read - Probability Theory for Non-Mathematicians
A plain-language guide to the ideas of probability that matter in business and survey work: the basic rules, conditional probability, Bayes' theorem, expected value, sample size and the intuitive traps that mislead decisions.
13 min read - The Law of Large Numbers and Its Practical Meaning
Why averages from larger random samples settle near the true value, how fast that happens, and why a large sample cannot rescue data that were collected with bias.
5 min read - The Central Limit Theorem Explained
What the central limit theorem says, why it lets analysts put margins of error on averages from skewed data, how large a sample really needs to be, and when the theorem fails.
12 min read
Descriptive statistics9 articles
- Measures of Central Tendency: Mean, Median and Mode
What the mean, median and mode each measure, how skewed data and outliers pull them apart, and how to choose the right "average" for survey and business reporting.
6 min read - Measures of Dispersion: Range, Variance and Standard Deviation
How the range, variance and standard deviation describe the spread of data, how to calculate and interpret them, and when robust alternatives serve survey and business data better.
10 min read - Percentiles, Quartiles and the Interquartile Range
What percentiles, quartiles and the interquartile range measure, why software gives different answers, how box plots and the 1.5 × IQR rule work, and how to use them with survey data.
8 min read - Skewness and Kurtosis: Describing the Shape of a Distribution
What skewness and kurtosis measure, how to calculate and read them, and why the shape of a distribution matters when reporting averages from survey and business data.
4 min read - Outliers: Detection, Causes and Treatment
What outliers are, why they arise, how to detect them with robust rules rather than fragile ones, and how to decide whether to correct, keep, down-weight or set aside an unusual value.
11 min read - Frequency Tables and Cross-Tabulations in Survey Analysis
How to build and read frequency tables and cross-tabulations, which percentages to report, how to test for association, and how to avoid the mistakes that most often mislead survey readers.
8 min read - Weighted Means and Weighted Percentages
How weighted means and weighted percentages work, why surveys need them, and how to check that weighting has not cost more precision than it gained.
3 min read - Index Numbers and Composite Indicators
How price and quantity index numbers summarise change over time, how composite indicators combine many measures into one score, and how choices of formula, weights and aggregation shape the results.
5 min read - Robust Statistics for Messy and Outlier-Prone Data
Why one bad value can wreck a mean, and how medians, trimmed means, robust spreads and robust outlier rules give stable answers from survey and business data that are less than perfectly clean.
8 min read
Distributions6 articles
- The Normal Distribution and Why It Appears Everywhere
What the bell-shaped normal distribution is, why sums and averages of many small effects tend to follow it, and when business and survey data do not.
4 min read - Binomial and Poisson Distributions in Business Applications
How the binomial and Poisson distributions describe yes/no outcomes and event counts, with worked business examples, the link between them, and the mistakes that most often mislead analysts.
7 min read - Long-Tailed and Power-Law Distributions
Why customer spend, product sales and many other business quantities have long tails, what power laws are, how to test for them properly, and how to summarise data dominated by a few extreme values.
9 min read - Sampling Distributions and Standard Errors
Why every survey estimate would change if the sample were drawn again, how the standard error measures that change, and how to calculate, report and interpret it correctly in business research.
10 min read - Kernel Density Estimation and Nonparametric Distribution Knowledge
How kernel density estimation draws a smooth picture of a distribution without assuming its shape, why the bandwidth matters most, and how to use it well on survey and business data.
5 min read - Extreme Value Theory Applications for Tail Risk Knowledge
How extreme value theory estimates the size and frequency of rare, damaging events, from block maxima to peaks over threshold, with a worked insurance example and practical guidance for business risk.
11 min read
Inference14 articles
- Confidence Intervals: Meaning and Misinterpretation
What a confidence interval actually promises, why even experienced researchers misread it, and how to report intervals from surveys and business data so that decision-makers take the right message from them.
9 min read - The Margin of Error in Survey Results
How the "plus or minus" figure on a survey is calculated, what drives it, what it leaves out, and how to report and plan for it in market research and business surveys.
11 min read - Hypothesis Testing: Null, Alternative and Decision Rules
How a statistical test turns a business question into a null and an alternative hypothesis, a decision rule and two kinds of error, with a worked survey example and practical guidance.
5 min read - P-Values: What They Mean and What They Do Not
What a p-value actually measures, the most common ways it is misread in business and survey reports, and how to report test results so that decision-makers are not misled.
3 min read - Type I and Type II Errors and Statistical Power
Every significance test can go wrong in two ways. This article explains false positives, false negatives and statistical power, and shows why underpowered surveys and tests mislead decision-makers.
4 min read - Power Analysis for Planning Studies
Power analysis tells you how many respondents a survey or experiment needs before fieldwork begins. This guide explains the inputs, works through a survey example and covers the most common planning mistakes.
5 min read - Effect Sizes: Reporting What Matters Beyond Significance
A p-value says whether a difference is distinguishable from noise, not how big it is. This article explains the main effect sizes, how to calculate and read them, and how to report them.
5 min read - Statistical Significance Versus Practical Significance
A result can be statistically significant yet too small to matter, or important yet not significant. This article explains the difference and shows how to judge findings by size, uncertainty and business value.
10 min read - Multiple Comparisons and the Problem of False Discoveries
Why running many significance tests on one dataset produces false positives, how Bonferroni, Holm and false discovery rate methods control them, and when adjustment is and is not needed.
5 min read - The ASA Statement on P-Values and the Move Beyond Significance
What the American Statistical Association said about p-values in 2016, why the debate went further in 2019, and how business and survey researchers can report evidence without relying on "significant".
11 min read - Measurement Error and Its Effects on Statistical Estimates
How random and systematic measurement error distort means, correlations, regression slopes and percentages in survey and business data, and what analysts can do to detect and correct it.
4 min read - Uncertainty Analysis for Estimates, Forecasts, and Models
How to identify, quantify and communicate the uncertainty in survey estimates, forecasts and models, with worked examples on sampling error, prediction intervals and Monte Carlo propagation.
7 min read - Robustness Checks and Sensitivity Analyses in Strengthening Knowledge Claims
How robustness checks, specification curves and sensitivity analyses show whether a finding survives reasonable changes to the analysis, with a worked loyalty-programme example and practical guidance for business data.
7 min read - Missing Data Mechanisms and Their Implications for Knowledge Completeness
Why the reason data are missing matters more than the amount, how MCAR, MAR and MNAR differ, and which analysis choices keep survey and business conclusions trustworthy.
7 min read
Tests and models40 articles
- The t-Test: Comparing Two Means
How the t-test compares two averages, when to use the one-sample, paired, Student and Welch versions, a worked advertising example, and how to report effect sizes and intervals rather than p-values alone.
10 min read - Analysis of Variance (ANOVA) and Its Extensions
How analysis of variance compares the means of three or more groups, how to read the F ratio and effect sizes, and when to use Welch, factorial, repeated-measures and covariance extensions.
5 min read - Chi-Square Tests for Categorical Survey Data
How chi-square tests check whether survey answers differ between groups or match an expected pattern, how to read expected counts, residuals and Cramér's V, and how to adjust for weighted samples.
5 min read - Non-Parametric Tests: Mann-Whitney, Wilcoxon and Kruskal-Wallis
How rank-based tests compare groups when data are skewed, ordinal or full of outliers, what the Mann-Whitney, Wilcoxon and Kruskal-Wallis tests actually test, and how to report them well.
10 min read - Correlation: Pearson, Spearman and Their Limits
How Pearson's and Spearman's correlation coefficients measure association, how to read and test them in survey data, and the outliers, curves, restricted ranges and other traps that mislead.
13 min read - Correlation Is Not Causation: Spurious Relationships in Data
The main ways a correlation can appear without a causal link, with classic examples, a visual guide to Simpson's paradox, and practical checks before acting on a relationship in data.
10 min read - Simple and Multiple Linear Regression
How simple and multiple linear regression describe a numeric outcome, how to read coefficients and R², which assumptions matter most, and how to apply regression soundly to survey and business data.
9 min read - Logistic Regression for Binary Outcomes
How logistic regression models yes/no outcomes such as renewal or purchase, how to read its odds ratios and predicted probabilities, and which mistakes to avoid in survey data.
3 min read - Ordinal and Multinomial Regression for Survey Responses
How ordinal and multinomial regression model survey answers with more than two categories, how to read their results as predicted probabilities, and which assumptions to check before reporting them.
4 min read - Multilevel and Hierarchical Models
How multilevel models handle data in which customers sit within stores, employees within teams or answers within respondents, and why they give fairer estimates for small groups.
4 min read - Time Series Analysis and Forecasting
How to describe, model and forecast data collected over time, from trend and seasonality to exponential smoothing, ARIMA, forecast accuracy and honest prediction intervals for business planning.
9 min read - Survival Analysis and Time-to-Event Data
How survival analysis measures how long it takes for something to happen, such as a customer cancelling, and why Kaplan–Meier curves and Cox models handle incomplete follow-up correctly.
6 min read - Structural Equation Modelling
How structural equation modelling links survey items to latent constructs and tests the paths between them, how to judge model fit, and which pitfalls to avoid in market research.
13 min read - Factor Analysis: Exploratory and Confirmatory
How exploratory and confirmatory factor analysis uncover and test the dimensions behind survey questions, how to choose the number of factors, and how to avoid the most common mistakes.
11 min read - Cluster Analysis for Market Segmentation
How cluster analysis groups customers into segments from survey data, how k-means and hierarchical methods work, how to choose the number of segments, and how to judge whether they are real and useful.
13 min read - Principal Component Analysis and Dimension Reduction
How principal component analysis condenses many correlated survey questions into a few summary dimensions, how to decide how many to keep, and where the method helps or misleads.
7 min read - Latent Class Analysis
How latent class analysis finds hidden groups in categorical survey answers, how to choose the number of classes, and how to profile, validate and report segments responsibly.
12 min read - Driver Analysis: Identifying What Influences Satisfaction
How driver analysis estimates which service attributes matter most for overall satisfaction, why ordinary regression weights mislead, and how to turn importance into sound priorities.
6 min read - Spatial Statistics and Geographic Information Systems in Knowledge Generation
How spatial statistics and geographic information systems turn location into evidence, how to measure and test geographic clustering, and which traps to avoid when mapping survey and business data.
4 min read - Network Analysis: Understanding Connectivity as a Source of Knowledge
How network analysis turns relationships between people, firms and products into evidence, from centrality and communities to statistical network models, with practical guidance for survey and business data.
9 min read - Statistical Modeling of Complex Systems: Abstracting Knowledge from Noise
How statisticians model markets, economies and other complex systems, separating stable structure from noise without overfitting, and what the limits of such models mean for business decisions.
7 min read - Generalized Linear Models and Their Extensions for Diverse Outcome Types
How generalised linear models handle counts, yes/no answers, proportions, ratings and skewed spending in one framework, and which extensions to use when the basic model is not enough.
13 min read - Quantile Regression for Distributional Knowledge Beyond Mean Effects
How quantile regression shows what drives low, typical and high values of an outcome, why that matters for spending, prices and service times, and how to fit and report it well.
9 min read - Nonparametric Regression and Smoothing Techniques for Flexible Knowledge Representation
How smoothers such as LOESS, splines and generalised additive models let the data show the shape of a relationship, and how to choose the amount of smoothing without chasing noise.
3 min read - Diagnostic Checking and Residual Analysis in Statistical Knowledge Validation
Why a fitted model is not finished until its residuals have been examined, which plots and tests reveal problems, and how to respond when the diagnostics fail.
4 min read - Categorical Data Analysis and Log-Linear Models for Contingency Knowledge
How log-linear models extend the chi-square test to tables with three or more variables, separate real associations from spurious ones, and support sound conclusions from survey cross-tabulations.
6 min read - Functional Data Analysis for Curve and Trajectory Knowledge
How functional data analysis treats each customer journey, tracking series or growth path as a single curve, and how smoothing and functional principal components reveal the shapes that matter.
5 min read - Copula Methods for Modeling Dependence Structures in Multivariate Knowledge
How copulas separate the behaviour of each variable from the way variables move together, why a single correlation can hide joint extremes, and how to use copulas with survey and business data.
12 min read - Change-Point Detection Methods for Identifying Shifts in Knowledge Processes
How change-point methods find the moments when a tracking score, sales series or process measure shifts, how offline and online approaches differ, and how to avoid mistaking noise for change.
11 min read - Meta-Regression Techniques for Explaining Heterogeneity in Aggregated Knowledge
How meta-regression relates differences between study results to study characteristics, how to read its output, and why its findings are observational and easily over-interpreted.
5 min read - Handling Zero-Inflated and Overdispersed Data in Count Knowledge Models
Why real counts such as visits, purchases or complaints rarely fit a Poisson model, and how negative binomial, zero-inflated and hurdle models handle extra variation and excess zeros.
5 min read - Compositional Data Analysis for Proportional Knowledge Structures
Shares, budgets and mixes that sum to 100% carry only relative information. This article explains why standard statistics mislead with such data and how log-ratio methods analyse them properly.
11 min read - Hidden Markov Models for Latent State Sequence Knowledge
How hidden Markov models infer unobserved states such as customer engagement or economic regimes from sequences of observations, how they are estimated and decoded, and where they mislead.
10 min read - Dynamic Factor Models for High-Dimensional Time-Series Knowledge
How dynamic factor models summarise dozens or hundreds of related time series with a few common factors, and how they support nowcasting, composite indices and tracking studies.
13 min read - Joint Modeling of Longitudinal and Survival Outcomes for Integrated Knowledge
How joint models link repeated measurements, such as engagement scores or biomarkers, to the timing of events such as churn or illness, and why analysing the two separately can mislead.
9 min read - Spatial Autoregressive Models for Dependent Geographic Knowledge
How spatial autoregressive models handle data from neighbouring regions that are not independent, how to read their spillover effects, and when simpler approaches are enough.
6 min read - Multistate Models for Transition Process Knowledge
How multistate models describe movement between conditions over time, from trial to paying customer to churn or from health to illness, and how to estimate, check and use them.
12 min read - Graphical Models for Conditional Independence Knowledge Structures
How graphical models use pictures of nodes and edges to describe which variables are directly linked and which are independent once others are known, and how to use them in survey and market research.
7 min read - Wavelet Analysis for Multi-Scale Temporal Knowledge
How wavelet analysis shows which cycles are present in a time series and when they appear or fade, why it complements Fourier analysis, and how to use it on business and survey data.
4 min read - Spatial Point Process Models for Event Location Knowledge
How spatial point process models analyse the locations of events such as crimes, store openings, disease cases or customer addresses, telling apart random scatter, clustering and regular spacing, and linking event intensity to local conditions.
12 min read
Survey statistics7 articles
- Variance Estimation in Complex Survey Samples
Why textbook standard errors understate uncertainty in stratified, clustered and weighted surveys, and how linearisation, replicate weights and the bootstrap give honest margins of error.
12 min read - Small Area Estimation
How small area estimation borrows strength from models and auxiliary data to produce reliable figures for regions, districts and subgroups where survey samples are too small for direct estimates.
12 min read - Statistical Analysis of Weighted Survey Data
How survey weights, strata and clusters change standard errors, how design effects and replication methods measure the change, and how to analyse subgroups and fit models correctly.
7 min read - Analysing Likert Scale Data: Ordinal or Interval?
Whether agree–disagree ratings can be averaged has been argued for seventy years. This article explains both positions, the evidence behind them, and a practical way to analyse and report Likert data.
10 min read - Reliability Statistics: Cronbach's Alpha and Alternatives
What Cronbach's alpha measures, why it is often misread, and when coefficient omega, test–retest correlations or intraclass correlations give a better answer for survey scales.
8 min read - Inter-Rater Reliability: Cohen's Kappa and Related Measures
How to check whether two or more people code survey answers consistently, how Cohen's kappa corrects for chance agreement, why it can mislead, and which related measures to use instead.
12 min read - Significance Testing in Cross-Tabulated Survey Results
How the significance letters and arrows in survey banner tables are produced, why weighting and multiple comparisons change them, and how to report differences between subgroups honestly.
11 min read
Bayesian and computational14 articles
- Bayes' Theorem and Updating Beliefs With Evidence
How Bayes' theorem turns prior beliefs and new evidence into updated probabilities, shown through survey examples, with guidance on likelihood ratios, priors, credible intervals and common reasoning errors.
12 min read - Bayesian Inference in Market Research
How Bayesian inference combines prior knowledge with new survey data, why it suits small samples and concept tests, and how to choose priors and report results honestly.
3 min read - Bootstrap Methods and Resampling
How the bootstrap uses the data themselves to measure uncertainty for medians, ratios and model outputs, which intervals to trust, how to adapt it to surveys, and where it fails.
9 min read - Monte Carlo Simulation for Risk and Uncertainty
How Monte Carlo simulation turns uncertain inputs such as market size, uptake and cost into a full range of outcomes, with a worked launch example, guidance on runs and inputs, and common pitfalls.
6 min read - Multilevel Regression and Post-Stratification (MRP)
How MRP turns a national or unrepresentative survey into reliable estimates for provinces, regions and small segments, with a worked example, practical guidance and the mistakes to avoid.
8 min read - Bayesian Hierarchical Models for Partial Pooling and Knowledge Stabilization
How Bayesian hierarchical models let many small estimates borrow strength from one another, why this stabilises rankings and trends, and how to fit, check and report them.
8 min read - Simulation Studies for Evaluating Statistical Method Performance
How simulation studies test statistical methods on data where the truth is known, which performance measures to use, how many repetitions are enough and how to report results honestly.
8 min read - Bayesian Model Averaging for Uncertainty-Aware Knowledge Aggregation
How Bayesian model averaging combines the evidence from many plausible models, weighting each by its posterior probability, so that conclusions and forecasts reflect uncertainty about the model itself.
3 min read - Bayesian Networks for Probabilistic Knowledge Representation
How Bayesian networks represent uncertain knowledge as a graph of variables and conditional probabilities, how they update beliefs when evidence arrives, and how to build and use them responsibly with survey and business data.
8 min read - Gaussian Process Models for Flexible Nonlinear Knowledge Representation
How Gaussian process models learn smooth nonlinear relationships without a fixed formula, report honest uncertainty that grows where data are thin, and support tracking, mapping, forecasting and experimentation in business data.
12 min read - Dirichlet Process Mixtures for Nonparametric Clustering Knowledge
How Dirichlet process mixture models let the data decide how many clusters there are, how they are fitted, and how to use them responsibly for customer and survey segmentation.
10 min read - Approximate Bayesian Computation for Complex Model Knowledge
How approximate Bayesian computation estimates the parameters of models that can be simulated but whose likelihood cannot be written down, and what tolerance and summary statistics mean for the answer.
5 min read - Variational Inference Methods for Scalable Bayesian Knowledge
How variational inference turns Bayesian computation into optimisation so that models can be fitted to very large data sets, and what is lost in the approximation.
11 min read - Ensemble Kalman Filters for Sequential Data Assimilation Knowledge
How the ensemble Kalman filter combines a model's forecast with each new batch of data to track quantities that cannot be observed directly, from weather to market indicators, and what can go wrong.
9 min read
Causal inference18 articles
- Causal Inference: From Association to Cause
What it means to say one thing causes another, why simple comparisons mislead, and how experiments, natural experiments and careful adjustment turn associations in business data into credible causal estimates.
12 min read - Confounding Variables and How to Control for Them
What makes a variable a confounder, which variables should and should not be adjusted for, and how stratification, regression, matching and propensity-score weighting control confounding in survey and business data.
11 min read - Randomised Controlled Trials as the Benchmark for Causality
Why random assignment gives the most credible answer to cause-and-effect questions, what a well-run trial needs, and where randomised experiments reach their limits in business and survey research.
5 min read - Propensity Score Matching
How propensity score matching builds comparable groups from observational data, how to check that it worked, and why it cannot fix confounders that were never measured.
8 min read - Difference-in-Differences Analysis
How difference-in-differences compares changes over time in treated and untreated groups to estimate causal effects, why parallel trends matter, and what recent research says about staggered roll-outs.
13 min read - Regression Discontinuity Designs
How regression discontinuity designs use cut-off rules, such as spend thresholds and eligibility scores, to estimate causal effects, and how to check that the comparison at the threshold is credible.
5 min read - Instrumental Variables
How instrumental variables recover causal effects when hidden confounders bias ordinary regression, which assumptions they rest on, how to test instrument strength, and how to read the results.
12 min read - Directed Acyclic Graphs for Causal Reasoning
How directed acyclic graphs turn assumptions about cause and effect into a simple diagram that shows which variables to adjust for, and which to leave alone, in business and survey analysis.
4 min read - Causal Mediation Analysis for Decomposing Pathways of Knowledge Influence
How causal mediation analysis splits an effect into the part that works through knowledge or another intermediate variable and the part that does not, which assumptions that needs, and how to report it.
8 min read - Interrupted Time-Series Designs for Intervention Knowledge Evaluation
How interrupted time-series designs use repeated measurements before and after a launch, policy or price change to estimate its effect, with segmented regression, a worked tracker example and the main pitfalls.
9 min read - Sensitivity Analysis for Unmeasured Confounding in Causal Knowledge
How sensitivity analysis asks how strong an unmeasured confounder would have to be to overturn an observational finding, with E-values, partial R² methods, a worked churn example and reporting guidance.
9 min read - Targeted Maximum Likelihood Estimation for Causal Knowledge from Observational Data
How targeted maximum likelihood estimation combines an outcome model and a propensity score to estimate causal effects from observational data, with a worked survey example, assumptions and practical cautions.
7 min read - Causal Forests and Heterogeneous Treatment Effect Estimation
How causal forests adapt random forests to estimate how a treatment effect varies from customer to customer, why honest estimation matters for valid intervals, and how to use the results for targeting.
10 min read - Synthetic Control Methods for Comparative Case Knowledge
How the synthetic control method builds a weighted comparison unit to estimate the effect of an intervention on a single region, market or organisation, and how to judge whether the result is credible.
12 min read - Double Machine Learning for Debiased Causal Estimation
How double machine learning uses flexible prediction models to adjust for many confounders while still delivering an unbiased causal estimate with a valid confidence interval, and when it can be trusted.
10 min read - Causal Discovery Algorithms from Observational Data
How algorithms such as PC, GES, FCI and LiNGAM try to learn cause-and-effect structure from observational data, what assumptions they need, and how to use their output responsibly.
7 min read - Marginal Structural Models for Time-Varying Confounding
Why ordinary regression fails when treatments and confounders change over time, and how marginal structural models use inverse probability weights to estimate the effect of a treatment strategy.
4 min read - Difference-in-Differences Designs with Multiple Periods and Groups
Why the familiar two-way fixed effects regression can mislead when a policy or product is rolled out at different times, and which newer difference-in-differences estimators to use instead.
4 min read
Data science and machine learning16 articles
- Statistics and Machine Learning: Differences and Overlaps
How statistics and machine learning differ in goals, culture and validation, where their methods now overlap, and how business and survey analysts can choose the right approach for each question.
10 min read - Predictive Modelling: Training, Testing and Validation
Why a predictive model must be judged on data it has not seen, how training, validation and test sets and cross-validation work, and the mistakes that make accuracy look better than it is.
3 min read - Overfitting and the Bias-Variance Trade-off
Why models that fit their data too closely predict badly, how prediction error splits into bias, variance and noise, and the practical tools that keep business models at the right level of complexity.
6 min read - Classification Models and Their Evaluation Metrics
A guide to the main classification models and to the metrics used to judge them, from the confusion matrix and accuracy to ROC curves, precision-recall, calibration and cost-based thresholds.
11 min read - Text Analytics and Natural Language Processing for Open-Ended Responses
How text analytics turns open-ended survey answers into reliable numbers, from word counts and topic models to supervised classifiers and language models, and how to validate the results.
6 min read - Sentiment Analysis: Methods and Accuracy
How sentiment analysis classifies opinions in reviews, comments and survey answers, how lexicon, machine learning and language model methods compare, and how to measure and correct their accuracy.
11 min read - Anomaly Detection in Business Data
How to find unusual values in sales, transactions, operations and survey data, from robust statistical rules and control charts to isolation forests, and how to separate real signals from errors and false alarms.
8 min read - The Ethics of Algorithmic Forecasting: Bias in Statistical Knowledge
How bias enters algorithmic forecasts through data, labels, history and use, why statistical fairness criteria conflict, and what responsible analysts can check before a score shapes decisions.
10 min read - Model Selection Criteria (AIC, BIC, Cross-Validation) and Their Epistemic Implications
How AIC, BIC and cross-validation choose between competing models, what each assumes about truth and prediction, and why the choice of criterion shapes what an analysis can honestly claim.
12 min read - Ensemble Methods and Model Averaging for Robust Knowledge Aggregation
How bagging, random forests, boosting, stacking and forecast combination pool many imperfect models into more accurate and stable predictions, and what that pooling can and cannot tell us.
7 min read - Penalized Regression (Lasso, Ridge, Elastic Net) for High-Dimensional Knowledge
How ridge, lasso and elastic net regression add a penalty to keep models with many predictors stable and predictive, how to tune them, and what they can and cannot tell decision-makers.
13 min read - Synthetic Minority Oversampling and Imbalanced Data Knowledge
What class imbalance does to predictive models, how SMOTE creates synthetic minority cases, and why calibration, sensible thresholds and honest validation often matter more than rebalancing.
8 min read - Conformal Prediction for Distribution-Free Uncertainty Knowledge
How conformal prediction wraps any predictive model in intervals with a guaranteed coverage rate, without distributional assumptions, and what that guarantee does and does not promise.
4 min read - Statistical Learning Theory Foundations and Their Relation to Knowledge Discovery
The core ideas of statistical learning theory, from risk and generalisation to capacity and the bias-variance trade-off, and what they mean for finding reliable patterns in data.
3 min read - Bayesian Additive Regression Trees for Flexible Predictive Knowledge
How Bayesian additive regression trees combine many small decision trees with a Bayesian prior to give flexible predictions with honest uncertainty, and why they are popular for estimating varied treatment effects.
8 min read - Transfer Learning Approaches in Statistical Modeling
How transfer learning uses data or models from a related setting to build better models where local data are scarce, which statistical forms it takes, and how to avoid transfer that makes results worse.
6 min read
Communication and misuse14 articles
- Simpson's Paradox: When Aggregated Data Mislead
Why a comparison that holds in every subgroup can reverse when the groups are combined, shown with the kidney stone data and a business experiment, and how to decide which figure to trust.
7 min read - Base Rate Neglect and the Misreading of Probabilities
Why accurate tests, models and screeners still produce mostly false alarms when the thing they look for is rare, how to correct the reasoning with Bayes' theorem, and how to present probabilities so they are understood.
9 min read - Survivorship Bias in Data and Decisions
How looking only at the cases that survived a selection process distorts averages, trends and lessons, with examples from aircraft, investment funds, customer cohorts and surveys, and practical checks against it.
7 min read - Regression to the Mean in Business Performance
Why star performers tend to slip and laggards tend to recover, how this statistical effect fools managers into crediting or blaming the wrong causes, and how to analyse performance data without being misled.
8 min read - How to Lie With Statistics: Common Distortions and How to Spot Them
A practical guide to the classic ways numbers mislead, from well-chosen averages and relative risks to biased samples, selective reporting and distorted charts, with questions any reader can ask.
10 min read - Misleading Graphs: Truncated Axes, Cherry-Picked Ranges and Dual Axes
Three of the most common ways charts mislead, each shown beside an honest version of the same data, with evidence on how readers are fooled and simple rules for drawing fair graphs.
6 min read - Principles of Data Visualisation for Statistical Results
How to choose and design charts that show statistical results accurately, drawing on research into graphical perception, colour and the display of uncertainty, with practical guidance for survey reports.
7 min read - Communicating Uncertainty to Non-Specialist Audiences
Practical, evidence-based ways to explain margins of error, ranges and risks to managers, clients and the public without hiding the uncertainty or burying the message.
3 min read - Statistical Literacy as a Business Skill
What statistical literacy means for managers and other non-specialists, which reasoning skills matter most in business, and how organisations can build the habit of questioning numbers well.
6 min read - Reproducible Statistical Analysis: Scripts, Versioning and Documentation
Why every published number should be traceable to raw data and code, and how scripts, version control and documentation make survey and business analyses checkable, repeatable and cheaper to run.
12 min read - Official Statistics: Independence, Trust and Quality Standards
What makes official statistics trustworthy, how international principles and codes protect their independence and quality, and what businesses using census, labour market and economic data should know.
9 min read - Data Privacy in Statistics: Disclosure Control and Differential Privacy
Why published tables and datasets can reveal individuals, how traditional disclosure control protects them, and what differential privacy adds, with a worked example and practical guidance for survey data.
7 min read - A Practical Framework for Statistical Quality Assurance
A stage-by-stage framework for checking statistical work, from planning to reporting, with practical checks, a worked example and the common failures that quality assurance is designed to catch.
6 min read - Statistical Process Control and Monitoring for Ongoing Knowledge Quality
How Shewhart control charts separate routine variation from real problems, how to build and read a p-chart, and how to monitor recurring surveys, data pipelines and business metrics without overreacting.
6 min read
Other disciplines
Explore the other disciplines
Inquiry
The art of asking good questions and reasoning carefully toward answers.
Insights
What makes a finding useful, and how teams turn data into decisions.
Surveys
How surveys are designed, sampled, fielded and checked, from the first questionnaires to online panels.
Research
The methods and standards behind sound research, from study design to reporting the results.
Statistics
The numbers behind the findings: describing data, testing ideas and reading results correctly.
Intelligence
Turning scattered information into knowledge an organisation can act on.
Investigation
Methods for establishing facts in business, markets, science and public life.
The Generation of Knowledge
How knowledge is created, tested, shared and trusted.