Skip to content

Surveys · Incentives and engagement

Prepaid Versus Promised Incentives: Evidence From Experiments

Incentives can be sent with the survey request or promised to people who complete it. This article reviews the experimental evidence comparing the two, explains why prepaid incentives usually work better and discusses when promised incentives make sense.

ModQu research article · 1,782 words · 19 references (APA 7th edition)

Download PDF

Survey designers who decide to offer an incentive face a basic choice about timing. A prepaid, or unconditional, incentive is given to everyone who is asked to take part, usually enclosed with the invitation letter, whether or not they respond. A promised, or conditional, incentive is given only to those who complete the survey. At first sight, promised incentives look more efficient: money goes only to people who respond. Yet decades of experiments show that prepaid incentives usually produce much larger increases in response, often enough to make them more cost-effective as well (Church, 1993; Singer & Ye, 2013).

This article reviews the experimental evidence, explains why prepaid incentives work better and discusses the situations in which promised incentives remain the practical choice.

The evidence from mail surveys

The clearest evidence comes from mail surveys, where randomised experiments are easy to run. Church (1993) carried out a meta-analysis of 38 experiments. Prepaid monetary incentives increased response rates by an average of about 19 percentage points compared with no incentive, while promised monetary incentives had small effects that were not reliably different from zero. Non-monetary gifts had smaller effects than money, and prepaid gifts again outperformed promised ones.

Individual experiments show the same pattern. Berk et al. (1987) compared prepaid and promised incentives in a controlled experiment and found the prepaid incentive more effective. James and Bolstein (1990) found that small prepaid cash amounts increased response in a mail survey and that the effect grew with the amount. In a later study, James and Bolstein (1992) found that a promised payment of $50 produced no more response than no incentive at all, while prepaid amounts as small as $1 increased response. Warriner et al. (1996) tested cash, lottery and charity incentives in a Canadian mail survey and found that only prepaid cash produced a clear gain. Edwards et al. (2002), in a systematic review of trials of postal questionnaires, confirmed that unconditional incentives were more effective than conditional ones.

Interviewer-administered surveys

Prepaid incentives also work in telephone and face-to-face surveys, provided there is a way to deliver them in advance, such as a letter sent to a known address. Singer et al. (1999) carried out a meta-analysis of incentive experiments in interviewer-mediated surveys and found that incentives increased response, that prepaid incentives were more effective than promised ones, and that money was more effective than gifts. Singer et al. (2000) found that a $5 bill enclosed with an advance letter increased response in a telephone survey, while a promised incentive of the same amount did not. Figure 1 summarises the typical pattern across modes.

Grouped bar chart of response rates with no incentive, a promised incentive and a prepaid incentive in mail, telephone and web surveys, with prepaid highest in each.
Figure 1. Response rates with no incentive, a promised incentive and a prepaid incentive in three modes (illustrative values). Chart by ModQu.

Web surveys

Web surveys present a practical difficulty. When invitations are sent by email, there is no easy way to include cash, and electronic vouchers or payment codes are less tangible. Bosnjak and Tuten (2003) compared prepaid and promised incentives in a web survey, using payments through an online payment service, and found that neither increased response much, while a prize draw performed somewhat better. Göritz (2006) found in a meta-analysis that incentives in web surveys had positive but modest effects.

When invitations to a web survey are sent by mail, however, the classic pattern returns. Millar and Dillman (2011) found that a prepaid cash incentive enclosed in a letter inviting people to respond online increased web response substantially. Messer and Dillman (2011) reported similar effects in address-based general population surveys using web-push designs. Prepaid cash sent by mail has become a standard part of these designs (Dillman et al., 2014).

Why prepaid incentives work better

Several explanations have been proposed. Figure 2 summarises four.

Four cards explaining reciprocity, trust, attention and certainty as reasons prepaid incentives work.
Figure 2. Four reasons why prepaid incentives work better than promised ones. Diagram by ModQu.

The most influential explanation is the norm of reciprocity: people feel obliged to return favours they have received (Gouldner, 1960). A prepaid incentive is a gift; responding is a way of returning it. A promised incentive is a payment for a service, which people can accept or decline on economic grounds, and small amounts are rarely enough to make the effort seem worth it.

Social exchange theory adds trust. By sending money in advance, the sponsor shows that it trusts the recipient and that the request is genuine, which makes people more willing to trust the sponsor in return (Dillman et al., 2014). A prepaid incentive also draws attention to the request: an envelope containing cash is more likely to be opened and its letter read. Finally, a prepaid incentive is certain, whereas people may doubt that a promised payment will actually arrive.

The size of the incentive matters less than its timing. Small prepaid amounts often outperform much larger promised ones, which fits the reciprocity explanation: the symbolic value of the gift matters more than its monetary value (James & Bolstein, 1992; Trussell & Lavrakas, 2004).

Cost-effectiveness

Because prepaid incentives are given to everyone, including nonrespondents, they can appear wasteful. Whether they are more cost-effective than promised incentives depends on how much they raise response and on the other costs of the survey. In mail and address-based surveys, a prepaid incentive often reduces the number of follow-up mailings, telephone calls or visits needed, so that the cost per completed questionnaire can be similar to or lower than without an incentive (Singer & Ye, 2013). Trussell and Lavrakas (2004) tested increasing prepaid amounts and found that response increased with each step, which allows designers to choose the amount that gives the best balance of cost and response.

Promised incentives can be more economical when the response rate is expected to be low and the incentive large, because a large prepaid incentive sent to many nonrespondents would be expensive.

Effects on who responds and on quality

Both kinds of incentive can change the composition of the responding sample. Prepaid incentives tend to have larger effects among groups that are otherwise less likely to respond, such as people with lower incomes or less interest in the topic, which can reduce nonresponse bias (Singer & Ye, 2013). Promised incentives may disproportionately attract people who are motivated by the payment itself.

Concerns that prepaid incentives might reduce data quality, by bringing in reluctant respondents who make little effort, have generally not been supported. Medway and Tourangeau (2015) found that prepaid cash incentives in a telephone survey did not reduce response quality and slightly reduced some forms of satisficing.

Lotteries and prize draws

Prize draws are a special form of promised incentive: respondents who complete the survey are entered into a draw for a larger prize. They are popular because the cost is fixed regardless of the number of respondents. The evidence suggests they have small effects. Warriner et al. (1996) found that a lottery did not increase response in a mail survey, while prepaid cash did. In online studies, LaRose and Tsai (2014) compared a sweepstakes with a prepaid cash incentive and found that the prepaid incentive produced higher completion rates and a sample closer to the population on some characteristics. Prize draws may be useful where no other incentive is feasible, but they should not be expected to match prepaid cash.

Prepaid and promised incentives in panel studies

Panel and longitudinal surveys combine both kinds of incentive over time. Members may receive a small unconditional payment with each wave's advance letter and a further payment when they complete the interview. Laurie and Lynn (2009) reviewed incentive practice in longitudinal surveys and found that unconditional incentives at each wave supported retention, and that members came to expect incentives once they had been introduced. Because expectations build up, changes from prepaid to promised incentives, or reductions in amount, can lower response in later waves. Panels that need to change their incentive scheme should test the change on a random subsample first.

When promised incentives make sense

Prepaid incentives are not always possible or appropriate. Promised incentives remain the practical choice in several situations:

Combinations are common: a small prepaid incentive to create goodwill, followed by a promised incentive on completion.

Ethical and practical considerations

Sending cash by mail raises practical questions. Some organisations are not permitted to send cash, and some recipients may not be able to use cheques or vouchers easily. Coins and small notes are sometimes stolen in the post, and enclosed money can make a letter look like a marketing promotion if the wording is not careful. A short sentence explaining that the money is a small token of thanks, and that the recipient may keep it whether or not they take part, avoids misunderstanding. Ethically, prepaid incentives are generally regarded as less coercive than large promised payments, because the recipient keeps the money whatever they decide (Singer & Ye, 2013).

Practical guidance

An example

A research company plans a mail-to-web survey of 4,000 households. A pilot compares three groups: no incentive, a $10 gift card promised on completion and a $2 coin enclosed with the letter. Response is 17 per cent with no incentive, 19 per cent with the promised gift card and 27 per cent with the enclosed coin. Although the prepaid incentive costs $8,000 for the full sample, it reduces the number of reminder mailings needed and produces the lowest cost per completed questionnaire (illustrative example).

Conclusion

The evidence from experiments is clear: incentives given in advance, unconditionally, increase response far more than incentives promised on completion, even when the promised amount is much larger. The difference reflects the social meaning of a gift and the trust it signals. Where postal contact is possible, small prepaid cash incentives are among the most effective tools available to survey designers. Promised incentives remain useful where advance payment is impossible, but they should be expected to have smaller effects.

References

Berk, M. L., Mathiowetz, N. A., Ward, E. P., & White, A. A. (1987). The effect of prepaid and promised incentives: Results of a controlled experiment. Journal of Official Statistics, 3(4), 449–457.

Bosnjak, M., & Tuten, T. L. (2003). Prepaid and promised incentives in web surveys: An experiment. Social Science Computer Review, 21(2), 208–217. https://doi.org/10.1177/0894439303021002006

Church, A. H. (1993). Estimating the effect of incentives on mail survey response rates: A meta-analysis. Public Opinion Quarterly, 57(1), 62–79. https://doi.org/10.1086/269355

Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method (4th ed.). Wiley.

Edwards, P., Roberts, I., Clarke, M., DiGuiseppi, C., Pratap, S., Wentz, R., & Kwan, I. (2002). Increasing response rates to postal questionnaires: Systematic review. BMJ, 324(7347), 1183. https://doi.org/10.1136/bmj.324.7347.1183

Gouldner, A. W. (1960). The norm of reciprocity: A preliminary statement. American Sociological Review, 25(2), 161–178. https://doi.org/10.2307/2092623

Göritz, A. S. (2006). Incentives in web studies: Methodological issues and a review. International Journal of Internet Science, 1(1), 58–70.

James, J. M., & Bolstein, R. (1990). The effect of monetary incentives and follow-up mailings on the response rate and response quality in mail surveys. Public Opinion Quarterly, 54(3), 346–361. https://doi.org/10.1086/269211

James, J. M., & Bolstein, R. (1992). Large monetary incentives and their effect on mail survey response rates. Public Opinion Quarterly, 56(4), 442–453. https://doi.org/10.1086/269336

LaRose, R., & Tsai, H.-Y. S. (2014). Completion rates and non-response error in online surveys: Comparing sweepstakes and pre-paid cash incentives in studies of online behavior. Computers in Human Behavior, 34, 110–119. https://doi.org/10.1016/j.chb.2014.01.017

Laurie, H., & Lynn, P. (2009). The use of respondent incentives on longitudinal surveys. In P. Lynn (Ed.), Methodology of longitudinal surveys (pp. 205–233). Wiley. https://doi.org/10.1002/9780470743874.ch12

Medway, R. L., & Tourangeau, R. (2015). Response quality in telephone surveys: Do prepaid cash incentives make a difference? Public Opinion Quarterly, 79(2), 524–543. https://doi.org/10.1093/poq/nfv011

Messer, B. L., & Dillman, D. A. (2011). Surveying the general public over the internet using address-based sampling and mail contact procedures. Public Opinion Quarterly, 75(3), 429–457. https://doi.org/10.1093/poq/nfr021

Millar, M. M., & Dillman, D. A. (2011). Improving response to web and mixed-mode surveys. Public Opinion Quarterly, 75(2), 249–269. https://doi.org/10.1093/poq/nfr003

Singer, E., & Ye, C. (2013). The use and effects of incentives in surveys. The Annals of the American Academy of Political and Social Science, 645(1), 112–141. https://doi.org/10.1177/0002716212458082

Singer, E., Van Hoewyk, J., & Maher, M. P. (2000). Experiments with incentives in telephone surveys. Public Opinion Quarterly, 64(2), 171–188. https://doi.org/10.1086/317761

Singer, E., Van Hoewyk, J., Gebler, N., Raghunathan, T., & McGonagle, K. (1999). The effect of incentives on response rates in interviewer-mediated surveys. Journal of Official Statistics, 15(2), 217–230.

Trussell, N., & Lavrakas, P. J. (2004). The influence of incremental increases in token cash incentives on mail survey response: Is there an optimal amount? Public Opinion Quarterly, 68(3), 349–367. https://doi.org/10.1093/poq/nfh022

Warriner, K., Goyder, J., Gjertsen, H., Hohner, P., & McSpurren, K. (1996). Charities, no; lotteries, no; cash, yes: Main effects and interactions in a Canadian incentives experiment. Public Opinion Quarterly, 60(4), 542–562. https://doi.org/10.1086/297772