Surveys · Incentives and engagement
Prepaid Versus Promised Incentives: Evidence From Experiments
Incentives can be sent with the survey request or promised to people who complete it. This article reviews the experimental evidence comparing the two, explains why prepaid incentives usually work better and discusses when promised incentives make sense.
Survey designers who decide to offer an incentive face a basic choice about timing. A prepaid, or unconditional, incentive is given to everyone who is asked to take part, usually enclosed with the invitation letter, whether or not they respond. A promised, or conditional, incentive is given only to those who complete the survey. At first sight, promised incentives look more efficient: money goes only to people who respond. Yet decades of experiments show that prepaid incentives usually produce much larger increases in response, often enough to make them more cost-effective as well (Church, 1993; Singer & Ye, 2013).
This article reviews the experimental evidence, explains why prepaid incentives work better and discusses the situations in which promised incentives remain the practical choice.
The evidence from mail surveys
The clearest evidence comes from mail surveys, where randomised experiments are easy to run. Church (1993) carried out a meta-analysis of 38 experiments. Prepaid monetary incentives increased response rates by an average of about 19 percentage points compared with no incentive, while promised monetary incentives had small effects that were not reliably different from zero. Non-monetary gifts had smaller effects than money, and prepaid gifts again outperformed promised ones.
Individual experiments show the same pattern. Berk et al. (1987) compared prepaid and promised incentives in a controlled experiment and found the prepaid incentive more effective. James and Bolstein (1990) found that small prepaid cash amounts increased response in a mail survey and that the effect grew with the amount. In a later study, James and Bolstein (1992) found that a promised payment of $50 produced no more response than no incentive at all, while prepaid amounts as small as $1 increased response. Warriner et al. (1996) tested cash, lottery and charity incentives in a Canadian mail survey and found that only prepaid cash produced a clear gain. Edwards et al. (2002), in a systematic review of trials of postal questionnaires, confirmed that unconditional incentives were more effective than conditional ones.
Interviewer-administered surveys
Prepaid incentives also work in telephone and face-to-face surveys, provided there is a way to deliver them in advance, such as a letter sent to a known address. Singer et al. (1999) carried out a meta-analysis of incentive experiments in interviewer-mediated surveys and found that incentives increased response, that prepaid incentives were more effective than promised ones, and that money was more effective than gifts. Singer et al. (2000) found that a $5 bill enclosed with an advance letter increased response in a telephone survey, while a promised incentive of the same amount did not. Figure 1 summarises the typical pattern across modes.
Web surveys
Web surveys present a practical difficulty. When invitations are sent by email, there is no easy way to include cash, and electronic vouchers or payment codes are less tangible. Bosnjak and Tuten (2003) compared prepaid and promised incentives in a web survey, using payments through an online payment service, and found that neither increased response much, while a prize draw performed somewhat better. Göritz (2006) found in a meta-analysis that incentives in web surveys had positive but modest effects.
When invitations to a web survey are sent by mail, however, the classic pattern returns. Millar and Dillman (2011) found that a prepaid cash incentive enclosed in a letter inviting people to respond online increased web response substantially. Messer and Dillman (2011) reported similar effects in address-based general population surveys using web-push designs. Prepaid cash sent by mail has become a standard part of these designs (Dillman et al., 2014).
Why prepaid incentives work better
Several explanations have been proposed. Figure 2 summarises four.
The most influential explanation is the norm of reciprocity: people feel obliged to return favours they have received (Gouldner, 1960). A prepaid incentive is a gift; responding is a way of returning it. A promised incentive is a payment for a service, which people can accept or decline on economic grounds, and small amounts are rarely enough to make the effort seem worth it.
Social exchange theory adds trust. By sending money in advance, the sponsor shows that it trusts the recipient and that the request is genuine, which makes people more willing to trust the sponsor in return (Dillman et al., 2014). A prepaid incentive also draws attention to the request: an envelope containing cash is more likely to be opened and its letter read. Finally, a prepaid incentive is certain, whereas people may doubt that a promised payment will actually arrive.
The size of the incentive matters less than its timing. Small prepaid amounts often outperform much larger promised ones, which fits the reciprocity explanation: the symbolic value of the gift matters more than its monetary value (James & Bolstein, 1992; Trussell & Lavrakas, 2004).
Cost-effectiveness
Because prepaid incentives are given to everyone, including nonrespondents, they can appear wasteful. Whether they are more cost-effective than promised incentives depends on how much they raise response and on the other costs of the survey. In mail and address-based surveys, a prepaid incentive often reduces the number of follow-up mailings, telephone calls or visits needed, so that the cost per completed questionnaire can be similar to or lower than without an incentive (Singer & Ye, 2013). Trussell and Lavrakas (2004) tested increasing prepaid amounts and found that response increased with each step, which allows designers to choose the amount that gives the best balance of cost and response.
Promised incentives can be more economical when the response rate is expected to be low and the incentive large, because a large prepaid incentive sent to many nonrespondents would be expensive.
Effects on who responds and on quality
Both kinds of incentive can change the composition of the responding sample. Prepaid incentives tend to have larger effects among groups that are otherwise less likely to respond, such as people with lower incomes or less interest in the topic, which can reduce nonresponse bias (Singer & Ye, 2013). Promised incentives may disproportionately attract people who are motivated by the payment itself.
Concerns that prepaid incentives might reduce data quality, by bringing in reluctant respondents who make little effort, have generally not been supported. Medway and Tourangeau (2015) found that prepaid cash incentives in a telephone survey did not reduce response quality and slightly reduced some forms of satisficing.
Lotteries and prize draws
Prize draws are a special form of promised incentive: respondents who complete the survey are entered into a draw for a larger prize. They are popular because the cost is fixed regardless of the number of respondents. The evidence suggests they have small effects. Warriner et al. (1996) found that a lottery did not increase response in a mail survey, while prepaid cash did. In online studies, LaRose and Tsai (2014) compared a sweepstakes with a prepaid cash incentive and found that the prepaid incentive produced higher completion rates and a sample closer to the population on some characteristics. Prize draws may be useful where no other incentive is feasible, but they should not be expected to match prepaid cash.
Prepaid and promised incentives in panel studies
Panel and longitudinal surveys combine both kinds of incentive over time. Members may receive a small unconditional payment with each wave's advance letter and a further payment when they complete the interview. Laurie and Lynn (2009) reviewed incentive practice in longitudinal surveys and found that unconditional incentives at each wave supported retention, and that members came to expect incentives once they had been introduced. Because expectations build up, changes from prepaid to promised incentives, or reductions in amount, can lower response in later waves. Panels that need to change their incentive scheme should test the change on a random subsample first.
When promised incentives make sense
Prepaid incentives are not always possible or appropriate. Promised incentives remain the practical choice in several situations:
- Email or online recruitment, where there is no postal address and cash cannot be enclosed.
- Online panels, where members are paid in points for each completed survey and the payment system is built around completion.
- Long or demanding tasks, such as diaries, multiple waves or physical measurements, where a promised payment compensates for substantial effort. These are often combined with a small prepaid token.
- Very large incentives, where sending the full amount to everyone would be too costly.
Combinations are common: a small prepaid incentive to create goodwill, followed by a promised incentive on completion.
Ethical and practical considerations
Sending cash by mail raises practical questions. Some organisations are not permitted to send cash, and some recipients may not be able to use cheques or vouchers easily. Coins and small notes are sometimes stolen in the post, and enclosed money can make a letter look like a marketing promotion if the wording is not careful. A short sentence explaining that the money is a small token of thanks, and that the recipient may keep it whether or not they take part, avoids misunderstanding. Ethically, prepaid incentives are generally regarded as less coercive than large promised payments, because the recipient keeps the money whatever they decide (Singer & Ye, 2013).
Practical guidance
- Where a postal address is available, enclose a small cash incentive with the first request.
- Choose cash rather than vouchers, gifts or lotteries when possible.
- Mention the incentive in the letter as a token of thanks, not as payment.
- Where only promised incentives are possible, make the payment simple and quick to receive, and state clearly when and how it will be made.
- Test the amount and type of incentive experimentally when the survey is large or repeated.
An example
A research company plans a mail-to-web survey of 4,000 households. A pilot compares three groups: no incentive, a $10 gift card promised on completion and a $2 coin enclosed with the letter. Response is 17 per cent with no incentive, 19 per cent with the promised gift card and 27 per cent with the enclosed coin. Although the prepaid incentive costs $8,000 for the full sample, it reduces the number of reminder mailings needed and produces the lowest cost per completed questionnaire (illustrative example).
Conclusion
The evidence from experiments is clear: incentives given in advance, unconditionally, increase response far more than incentives promised on completion, even when the promised amount is much larger. The difference reflects the social meaning of a gift and the trust it signals. Where postal contact is possible, small prepaid cash incentives are among the most effective tools available to survey designers. Promised incentives remain useful where advance payment is impossible, but they should be expected to have smaller effects.
References
Berk, M. L., Mathiowetz, N. A., Ward, E. P., & White, A. A. (1987). The effect of prepaid and promised incentives: Results of a controlled experiment. Journal of Official Statistics, 3(4), 449–457.
Bosnjak, M., & Tuten, T. L. (2003). Prepaid and promised incentives in web surveys: An experiment. Social Science Computer Review, 21(2), 208–217. https://doi.org/10.1177/0894439303021002006
Church, A. H. (1993). Estimating the effect of incentives on mail survey response rates: A meta-analysis. Public Opinion Quarterly, 57(1), 62–79. https://doi.org/10.1086/269355
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method (4th ed.). Wiley.
Edwards, P., Roberts, I., Clarke, M., DiGuiseppi, C., Pratap, S., Wentz, R., & Kwan, I. (2002). Increasing response rates to postal questionnaires: Systematic review. BMJ, 324(7347), 1183. https://doi.org/10.1136/bmj.324.7347.1183
Gouldner, A. W. (1960). The norm of reciprocity: A preliminary statement. American Sociological Review, 25(2), 161–178. https://doi.org/10.2307/2092623
Göritz, A. S. (2006). Incentives in web studies: Methodological issues and a review. International Journal of Internet Science, 1(1), 58–70.
James, J. M., & Bolstein, R. (1990). The effect of monetary incentives and follow-up mailings on the response rate and response quality in mail surveys. Public Opinion Quarterly, 54(3), 346–361. https://doi.org/10.1086/269211
James, J. M., & Bolstein, R. (1992). Large monetary incentives and their effect on mail survey response rates. Public Opinion Quarterly, 56(4), 442–453. https://doi.org/10.1086/269336
LaRose, R., & Tsai, H.-Y. S. (2014). Completion rates and non-response error in online surveys: Comparing sweepstakes and pre-paid cash incentives in studies of online behavior. Computers in Human Behavior, 34, 110–119. https://doi.org/10.1016/j.chb.2014.01.017
Laurie, H., & Lynn, P. (2009). The use of respondent incentives on longitudinal surveys. In P. Lynn (Ed.), Methodology of longitudinal surveys (pp. 205–233). Wiley. https://doi.org/10.1002/9780470743874.ch12
Medway, R. L., & Tourangeau, R. (2015). Response quality in telephone surveys: Do prepaid cash incentives make a difference? Public Opinion Quarterly, 79(2), 524–543. https://doi.org/10.1093/poq/nfv011
Messer, B. L., & Dillman, D. A. (2011). Surveying the general public over the internet using address-based sampling and mail contact procedures. Public Opinion Quarterly, 75(3), 429–457. https://doi.org/10.1093/poq/nfr021
Millar, M. M., & Dillman, D. A. (2011). Improving response to web and mixed-mode surveys. Public Opinion Quarterly, 75(2), 249–269. https://doi.org/10.1093/poq/nfr003
Singer, E., & Ye, C. (2013). The use and effects of incentives in surveys. The Annals of the American Academy of Political and Social Science, 645(1), 112–141. https://doi.org/10.1177/0002716212458082
Singer, E., Van Hoewyk, J., & Maher, M. P. (2000). Experiments with incentives in telephone surveys. Public Opinion Quarterly, 64(2), 171–188. https://doi.org/10.1086/317761
Singer, E., Van Hoewyk, J., Gebler, N., Raghunathan, T., & McGonagle, K. (1999). The effect of incentives on response rates in interviewer-mediated surveys. Journal of Official Statistics, 15(2), 217–230.
Trussell, N., & Lavrakas, P. J. (2004). The influence of incremental increases in token cash incentives on mail survey response: Is there an optimal amount? Public Opinion Quarterly, 68(3), 349–367. https://doi.org/10.1093/poq/nfh022
Warriner, K., Goyder, J., Gjertsen, H., Hohner, P., & McSpurren, K. (1996). Charities, no; lotteries, no; cash, yes: Main effects and interactions in a Canadian incentives experiment. Public Opinion Quarterly, 60(4), 542–562. https://doi.org/10.1086/297772