Skip to content

Statistics · Bayesian and computational

Approximate Bayesian Computation for Complex Model Knowledge

How approximate Bayesian computation estimates the parameters of models that can be simulated but whose likelihood cannot be written down, and what tolerance and summary statistics mean for the answer.

ModQu research article · 1,146 words · 12 references (APA 7th edition)

Download PDF

Bayesian inference updates beliefs about unknown parameters using the likelihood: the probability of the observed data under each possible parameter value. For many realistic models, that likelihood cannot be calculated. Agent-based models of consumer behaviour, simulations of word-of-mouth spreading through a network, queueing models of a contact centre and models of population genetics can all be run forward to generate data, yet no formula gives the probability of a particular outcome. Approximate Bayesian computation (ABC) offers a way through. It replaces the likelihood with simulation: parameter values that produce simulated data resembling the real data are kept, and the rest are discarded.

The basic idea

The logic goes back to Rubin (1984), who described Bayesian inference as a process of drawing parameters, simulating data and keeping the draws whose data match the observations. ABC was developed in population genetics, where Tavaré et al. (1997) and Pritchard et al. (1999) used it to infer evolutionary histories from DNA data, and Beaumont et al. (2002) gave the method its name and a regression adjustment that improved its accuracy.

The simplest version, ABC rejection, repeats three steps many times:

  1. Draw a parameter value θ from the prior distribution.
  2. Simulate a data set from the model using θ.
  3. Keep θ if a distance between summary statistics of the simulated and observed data is smaller than a tolerance ε.

The kept values approximate the posterior distribution. If the summaries capture all the relevant information and ε is zero, the approximation is exact; in practice, both choices introduce error (Marin et al., 2012).

Tolerance and summary statistics

Figure 1 shows the effect of tolerance in a case where the exact answer is known. A hypothetical website records 24 conversions among 150 visitors, and the conversion rate is given a uniform prior between 0 and 0.4. Accepting only simulations that reproduce exactly 24 conversions gives a posterior that matches the exact one, but only about 1.7% of simulations are accepted. Allowing a tolerance of plus or minus eight conversions accepts about 28% of simulations, but the resulting distribution is much wider: its standard deviation is about 0.044 rather than 0.030.

Histogram chart for a conversion rate estimated from 24 conversions among 150 visitors. Green bars show parameter values accepted by ABC rejection when simulated conversions must match the observed 24 exactly; they follow the exact posterior curve, drawn as a dark line, centred near 0.16. Orange outline bars show values accepted with a tolerance of plus or minus 8 conversions; this approximation is much wider, spreading from about 0.08 to 0.25, so a loose tolerance overstates uncertainty.
Figure 1. ABC rejection with an exact match reproduces the exact posterior; a loose tolerance accepts more simulations but gives a wider, less informative answer. Illustrative data; simulation by ModQu.

The trade-off is general: smaller tolerances give better approximations but need far more simulations. With complex data, exact matching is impossible, so the data are reduced to summary statistics such as means, variances, correlations or counts. Summaries that discard relevant information produce a posterior that is too wide or even centred in the wrong place. Fearnhead and Prangle (2012) proposed building summaries semi-automatically from regression on pilot simulations, an approach that targets the posterior mean of each parameter.

A further refinement adjusts the accepted values rather than throwing information away. Beaumont et al. (2002) regressed the accepted parameter values on the differences between simulated and observed summaries, then shifted each value to where it would have been if its summaries had matched the data exactly. This local regression adjustment allows a larger tolerance, and therefore fewer wasted simulations, while keeping much of the accuracy of a small one.

A worked example: calibrating a referral model

Consider a hypothetical subscription service that wants to understand how its customers recruit friends. An agent-based simulation represents 5,000 customers on a social network. Each month, a customer recommends the service to each friend with probability r, and a friend who receives a recommendation joins with probability q; there is also a base rate b of joining through advertising. Because recruitment depends on who knows whom and on the timing of earlier sign-ups, the probability of the observed monthly sign-up counts cannot be written down.

The analyst places priors on r, q and b, runs the simulation for 200,000 parameter draws, and summarises each simulated year by three statistics: total sign-ups, the share of sign-ups that came with a referral code, and the month-to-month growth rate in the second half of the year. Draws whose summaries fall within the closest 1% of the observed values are kept. In this illustration the accepted draws show that the base rate b is well determined, while r and q are strongly negatively correlated: the data can tell that the product r × q is about 0.02, but not whether customers recommend often to reluctant friends or rarely to eager ones. That is useful knowledge in itself. It tells the business that a survey asking customers how often they recommend the service would add information the sign-up data cannot supply.

More efficient algorithms

Rejection sampling wastes most simulations when the prior is broad. Marjoram et al. (2003) embedded ABC in Markov chain Monte Carlo, proposing new parameter values near accepted ones. Sequential Monte Carlo versions move a population of parameter values through a series of shrinking tolerances, reusing what was learned at each stage (Sisson et al., 2007; Beaumont et al., 2009). These methods can reduce the number of simulations needed by orders of magnitude. More recent simulation-based inference methods use neural networks to learn the likelihood or posterior from simulations (Cranmer et al., 2020).

Uses in business and research

ABC is suited to models that are built to mimic a process rather than to fit a convenient distribution.

Limitations

Key takeaways

Approximate Bayesian computation makes Bayesian inference possible for models that can be simulated but not written as a likelihood. Its accuracy depends on the tolerance and summary statistics, which should be chosen deliberately and tested. Used with care, ABC lets complex, realistic models of customers, markets and processes be calibrated to real data, with honest statements of uncertainty.

References

Beaumont, M. A., Cornuet, J.-M., Marin, J.-M., & Robert, C. P. (2009). Adaptive approximate Bayesian computation. Biometrika, 96(4), 983–990. https://doi.org/10.1093/biomet/asp052

Beaumont, M. A., Zhang, W., & Balding, D. J. (2002). Approximate Bayesian computation in population genetics. Genetics, 162(4), 2025–2035. https://doi.org/10.1093/genetics/162.4.2025

Cranmer, K., Brehmer, J., & Louppe, G. (2020). The frontier of simulation-based inference. Proceedings of the National Academy of Sciences, 117(48), 30055–30062. https://doi.org/10.1073/pnas.1912789117

Csilléry, K., Blum, M. G. B., Gaggiotti, O. E., & François, O. (2010). Approximate Bayesian computation (ABC) in practice. Trends in Ecology & Evolution, 25(7), 410–418. https://doi.org/10.1016/j.tree.2010.04.001

Fearnhead, P., & Prangle, D. (2012). Constructing summary statistics for approximate Bayesian computation: Semi-automatic approximate Bayesian computation. Journal of the Royal Statistical Society Series B: Statistical Methodology, 74(3), 419–474. https://doi.org/10.1111/j.1467-9868.2011.01010.x

Marin, J.-M., Pudlo, P., Robert, C. P., & Ryder, R. J. (2012). Approximate Bayesian computational methods. Statistics and Computing, 22(6), 1167–1180. https://doi.org/10.1007/s11222-011-9288-2

Marjoram, P., Molitor, J., Plagnol, V., & Tavaré, S. (2003). Markov chain Monte Carlo without likelihoods. Proceedings of the National Academy of Sciences, 100(26), 15324–15328. https://doi.org/10.1073/pnas.0306899100

Pritchard, J. K., Seielstad, M. T., Perez-Lezaun, A., & Feldman, M. W. (1999). Population growth of human Y chromosomes: A study of Y chromosome microsatellites. Molecular Biology and Evolution, 16(12), 1791–1798. https://doi.org/10.1093/oxfordjournals.molbev.a026091

Robert, C. P., Cornuet, J.-M., Marin, J.-M., & Pillai, N. S. (2011). Lack of confidence in approximate Bayesian computation model choice. Proceedings of the National Academy of Sciences, 108(37), 15112–15117. https://doi.org/10.1073/pnas.1102900108

Rubin, D. B. (1984). Bayesianly justifiable and relevant frequency calculations for the applied statistician. The Annals of Statistics, 12(4), 1151–1172. https://doi.org/10.1214/aos/1176346785

Sisson, S. A., Fan, Y., & Tanaka, M. M. (2007). Sequential Monte Carlo without likelihoods. Proceedings of the National Academy of Sciences, 104(6), 1760–1765. https://doi.org/10.1073/pnas.0607208104

Tavaré, S., Balding, D. J., Griffiths, R. C., & Donnelly, P. (1997). Inferring coalescence times from DNA sequence data. Genetics, 145(2), 505–518. https://doi.org/10.1093/genetics/145.2.505