Skip to content

Surveys · Sampling

Non-Probability Sampling: Convenience, Quota, Snowball and Purposive Methods

Non-probability samples are chosen without a random process that gives every population member a known chance of selection. This article explains the four main methods, what each is good for, and how their results can be made more trustworthy.

ModQu research article · 1,280 words · 13 references (APA 7th edition)

Download PDF

Not every survey uses a random sample. Many studies, from market research panels to qualitative interviews, select people in other ways: because they are easy to reach, because they fit a quota, because someone referred them or because the researcher chose them for a reason. These are non-probability samples. Because the chance that any given person is selected is unknown, the statistical theory that supports probability samples does not apply directly, and results cannot be generalised to a population with a known margin of error (Baker et al., 2013).

That does not make non-probability samples useless. They are often cheaper, faster and sometimes the only option. The question is what they can and cannot support. This article describes the four classic methods, shown in Figure 1, and what is known about their accuracy.

Four cards. Convenience sampling: whoever is easiest to reach, such as passers-by or website visitors; cheap and fast, with likely selection bias. Quota sampling: interviewers fill set numbers by age, sex, region or other traits; matches the population on those traits only. Snowball sampling: initial respondents refer others they know; reaches hidden groups but reflects social networks. Purposive sampling: the researcher chooses cases for a reason, such as experts or typical cases; suits depth, not prevalence.
Figure 1. Four non-probability sampling methods. Diagram by ModQu, based on Baker et al. (2013).

Convenience sampling

A convenience sample consists of people who are easy to reach: students in a lecture, shoppers in a mall, visitors to a website or volunteers who answer an open invitation. It is the cheapest and fastest method, which is why it is so common in pilot studies, classroom research and early product testing (Etikan et al., 2016).

The weakness is selection bias. The people who are easiest to reach usually differ from the rest of the population in ways that matter. Visitors to a website are more interested in its subject than people who never visit it; volunteers for a health survey are often healthier and better educated than average. Larger convenience samples do not solve this problem, because size does not correct a biased selection process (Baker et al., 2013).

Quota sampling

In quota sampling, the researcher sets targets for the number of respondents with particular characteristics, such as age, sex and region, usually in proportion to their share of the population. Interviewers or online systems then recruit people until each quota is full.

Quota sampling became the standard method of commercial and political polling in the mid-twentieth century. Its results match the population on the quota variables, which makes it look representative. Within each quota cell, however, selection is still by convenience. An experimental comparison of quota and random sampling in Britain found that quota samples could produce reasonable estimates for some variables but showed biases that their margins of error did not reflect (Moser & Stuart, 1953). Quota samples are as good as the assumption that, within each cell, the people recruited are similar to those who were not.

Snowball sampling

In snowball sampling, the researcher starts with a few members of the target group and asks them to recruit or name others, who in turn name others. The method was formalised as a statistical technique by Goodman (1961) and is widely used to reach groups for which no list exists, such as people who use drugs, members of informal networks or specialists in a narrow field (Biernacki & Waldorf, 1981).

Its strength is reach. Its weakness is that the sample reflects the social networks of the initial contacts, so people who are well connected are over-represented and isolated people may never be reached. Respondent-driven sampling was developed to address this, by tracking who recruited whom and using network information to weight the results (Heckathorn, 1997).

Purposive sampling

In purposive sampling, the researcher selects cases deliberately because they are informative for the research question: experts with specialist knowledge, typical cases, extreme cases or cases that differ in theoretically important ways. Purposive sampling is standard in qualitative research, where the goal is understanding rather than estimating how common something is (Patton, 2015; Palinkas et al., 2015).

It should not be used to estimate population figures. A purposive sample of ten industry experts can explain why a market is changing; it cannot say what share of firms hold a given view.

How accurate are non-probability samples?

Large comparisons of survey results with known benchmarks, such as government records, have generally found that probability samples are more accurate than non-probability online samples, and that the accuracy of non-probability samples varies more from one provider to another (Yeager et al., 2011; Cornesse et al., 2020). Non-probability samples are not always wrong, but it is harder to know in advance when they will be right.

Statistical adjustment can improve them. Weighting to match population totals, matching samples to a reference survey and modelling approaches such as MRP can reduce bias when the variables that drive selection are measured and included (Elliott & Valliant, 2017; Mercer et al., 2017). A well-known demonstration produced accurate election forecasts from a highly unrepresentative sample of video game users by modelling and adjusting the data (Wang et al., 2015). Adjustment works only to the extent that the reasons people end up in the sample are captured by the variables used.

In business research

Most commercial survey research today relies on non-probability samples, especially online panels whose members have signed up to take surveys in exchange for small rewards. Panel companies usually apply quotas on age, sex and region, so their samples combine convenience recruitment with quota control (Baker et al., 2013). For many business purposes, such as testing which of two advertisements people prefer or exploring reactions to a new product, this is adequate, because the comparison between versions matters more than an exact population figure. The risk is greatest when businesses use such samples to estimate the size of a market or the share of people with a given behaviour, because these figures depend directly on how representative the sample is.

An example

A company wants to know what share of small business owners in its region plan to buy new accounting software in the next year. It has three options. It could survey owners who visit its website, a convenience sample that would overstate interest because visitors are already looking at software. It could buy an online panel sample with quotas on business size and industry, which would be balanced on those traits but might over-represent owners who like taking surveys. Or it could draw a random sample from a business register, which would be slower and more expensive but would support a measurable margin of error. If the company only needs to compare reactions to two product designs, the panel sample may be good enough. If it needs a market size for an investment decision, the register sample is the safer choice (illustrative example).

When to use each method

Non-probability methods are appropriate when the goal is exploration, testing or understanding, rather than precise population estimates; when no frame exists; or when time and budget rule out probability sampling. In these situations, researchers should choose the method that best fits the purpose: purposive selection for depth, snowball recruitment for hidden groups, quotas for broad balance and convenience samples for quick tests.

Good practice

Conclusion

Non-probability sampling covers a wide range of practices, from quick convenience samples to carefully designed quota and network methods. Each has legitimate uses, and modern adjustment methods can make their estimates more accurate. But the absence of known selection probabilities means their accuracy rests on assumptions rather than design. Being clear about which method was used, and why, is the first step to using their results responsibly (Cornesse et al., 2020).

References

Baker, R., Brick, J. M., Bates, N. A., Battaglia, M., Couper, M. P., Dever, J. A., Gile, K. J., & Tourangeau, R. (2013). Summary report of the AAPOR task force on non-probability sampling. Journal of Survey Statistics and Methodology, 1(2), 90–143. https://doi.org/10.1093/jssam/smt008

Biernacki, P., & Waldorf, D. (1981). Snowball sampling: Problems and techniques of chain referral sampling. Sociological Methods & Research, 10(2), 141–163. https://doi.org/10.1177/004912418101000205

Cornesse, C., Blom, A. G., Dutwin, D., Krosnick, J. A., De Leeuw, E. D., Legleye, S., Pasek, J., Pennay, D., Phillips, B., Sakshaug, J. W., Struminskaya, B., & Wenz, A. (2020). A review of conceptual approaches and empirical evidence on probability and nonprobability sample survey research. Journal of Survey Statistics and Methodology, 8(1), 4–36. https://doi.org/10.1093/jssam/smz041

Elliott, M. R., & Valliant, R. (2017). Inference for nonprobability samples. Statistical Science, 32(2), 249–264. https://doi.org/10.1214/16-STS598

Etikan, I., Musa, S. A., & Alkassim, R. S. (2016). Comparison of convenience sampling and purposive sampling. American Journal of Theoretical and Applied Statistics, 5(1), 1–4. https://doi.org/10.11648/j.ajtas.20160501.11

Goodman, L. A. (1961). Snowball sampling. Annals of Mathematical Statistics, 32(1), 148–170. https://doi.org/10.1214/aoms/1177705148

Heckathorn, D. D. (1997). Respondent-driven sampling: A new approach to the study of hidden populations. Social Problems, 44(2), 174–199. https://doi.org/10.2307/3096941

Mercer, A. W., Kreuter, F., Keeter, S., & Stuart, E. A. (2017). Theory and practice in nonprobability surveys: Parallels between causal inference and survey inference. Public Opinion Quarterly, 81(S1), 250–271. https://doi.org/10.1093/poq/nfw060

Moser, C. A., & Stuart, A. (1953). An experimental study of quota sampling. Journal of the Royal Statistical Society: Series A (General), 116(4), 349–405. https://doi.org/10.2307/2343021

Palinkas, L. A., Horwitz, S. M., Green, C. A., Wisdom, J. P., Duan, N., & Hoagwood, K. (2015). Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 42(5), 533–544. https://doi.org/10.1007/s10488-013-0528-y

Patton, M. Q. (2015). Qualitative research and evaluation methods (4th ed.). SAGE.

Wang, W., Rothschild, D., Goel, S., & Gelman, A. (2015). Forecasting elections with non-representative polls. International Journal of Forecasting, 31(3), 980–991. https://doi.org/10.1016/j.ijforecast.2014.06.001

Yeager, D. S., Krosnick, J. A., Chang, L., Javitz, H. S., Levendusky, M. S., Simpser, A., & Wang, R. (2011). Comparing the accuracy of RDD telephone surveys and internet surveys conducted with probability and non-probability samples. Public Opinion Quarterly, 75(4), 709–747. https://doi.org/10.1093/poq/nfr020