Surveys · Incentives and engagement
Leverage-Saliency Theory and the Decision to Participate in a Survey
Leverage-saliency theory explains why the same survey request persuades some people and not others. This article sets out the theory, the experiments that support it and what it implies for survey design and nonresponse bias.
Why does one person agree to take part in a survey while a neighbour, receiving the same letter, throws it away? Earlier theories of survey participation focused on general factors, such as social exchange or civic duty, that apply to everyone. Leverage-saliency theory, proposed by Groves et al. (2000), starts instead from the observation that different people respond to different features of the same request. Some care about the topic, some about the sponsor, some about the incentive and some mainly about how long it will take.
The theory
Leverage-saliency theory describes the decision to participate as a weighing of the features, or attributes, of a survey request. Each attribute has two properties:
- Leverage is how much the attribute matters to a particular person, and in which direction. Interest in the topic has positive leverage for an enthusiast; the time required has negative leverage for someone who is busy.
- Saliency is how prominent the attribute is made in the request. A letter that opens by describing the topic makes the topic salient; one that leads with an incentive makes the incentive salient.
The decision depends on the combination: attributes with high leverage for a person push them towards or away from participation, and the more salient those attributes are, the stronger the push. Because leverage differs between people, the same request is weighed differently by each recipient (Groves et al., 2000). Figure 1 gives an example.
The theory builds on earlier work showing that people often decide quickly, using heuristic cues such as authority and reciprocity rather than careful deliberation (Groves et al., 1992; Groves & Couper, 1998). It adds the idea that the cues work differently for different people.
Evidence
Groves et al. (2000) tested the theory in an experiment in which interviewers' introductions emphasised different attributes. They found that people with high community involvement were more likely to respond to a survey about their community, and that an incentive increased response mainly among those with low involvement. In other words, the incentive compensated for the lack of interest.
Groves et al. (2004) sent survey requests on different topics to samples drawn from lists, such as teachers, new parents and political contributors. Members of each group were more likely to respond to a survey on a topic related to their group, confirming that topic interest has leverage. The effect was moderate in size: interest increased response but did not determine it. Groves et al. (2006) went further, showing experimentally that describing a survey's topic in the invitation could produce nonresponse bias in estimates related to that topic. Figure 2 illustrates the interaction between interest and incentives.
Implications for nonresponse bias
Leverage-saliency theory explains how nonresponse bias arises. If an attribute that drives participation is related to the survey's key variables, the responding sample will be distorted on those variables. A survey about volunteering that emphasises its topic will attract volunteers, overestimating volunteering in the population. The theory also explains why the response rate is a poor predictor of bias: what matters is not how many people respond but whether the reasons for responding are related to what is being measured (Groves, 2006; Groves & Peytcheva, 2008).
Implications for design
The theory suggests several practical approaches:
- Offer several reasons to take part. A request that combines a trusted sponsor, a general description of the topic, an incentive and a modest time commitment appeals to more kinds of people.
- Be careful about the topic's saliency. Describing the topic in detail can increase response among the interested and bias estimates related to that interest.
- Use incentives to balance interest. Because incentives have most leverage among people with little interest in the topic, they can reduce bias related to topic interest (Singer & Ye, 2013).
- Tailor follow-up. Interviewers, reminders and later contacts can emphasise different attributes for people who did not respond to the first appeal (Dillman et al., 2014).
Interviewers and tailoring
In interviewer surveys, the theory helps explain why skilled interviewers succeed. Groves and Couper (1998) described how experienced interviewers practise tailoring: they listen to the concerns a person raises at the doorstep or on the telephone and respond with the features of the survey most likely to matter to that person. To someone worried about time, the interviewer stresses that the interview is short; to someone suspicious of the sponsor, the interviewer explains who is conducting the research and why. They also aim at maintaining interaction, keeping the conversation going long enough to learn what matters to the person. In leverage-saliency terms, tailoring raises the saliency of attributes with positive leverage for the individual and addresses those with negative leverage. Standardised scripts, which present the same features to everyone, cannot do this.
Relation to other theories
Leverage-saliency theory complements rather than replaces other explanations of participation. Social exchange theory, which underlies the tailored design method, emphasises trust, rewards and costs in general terms (Dillman et al., 2014). Singer (2011) proposed a benefit-cost theory in which people weigh the perceived benefits of taking part, such as interest, helping others or an incentive, against perceived costs, such as time and risks to privacy. Leverage-saliency theory adds two ideas to these accounts: that the weights differ from person to person, and that the design of the request can change which features people notice.
Wider applications
The theory has been applied beyond interviewer surveys. Keusch (2015) reviewed participation in web surveys and found leverage-saliency theory useful for explaining why features such as the sender, the incentive and the topic affect response differently among different groups. In market research, the same logic applies to customer and employee surveys: a customer with a recent complaint and one with no strong views will respond to different parts of the same invitation, which is one reason satisfaction surveys can over-represent people with strong opinions.
Conclusion
Leverage-saliency theory treats the decision to take part in a survey as a personal weighing of the features of a request. Because people weigh features differently, a request that emphasises one feature appeals to some and not others, which affects both response rates and the composition of the sample. The theory gives survey designers a practical framework: present several reasons to take part, avoid making the topic too prominent and use incentives to bring in the less interested.
References
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method (4th ed.). Wiley.
Groves, R. M., & Couper, M. P. (1998). Nonresponse in household interview surveys. Wiley. https://doi.org/10.1002/9781118490082
Groves, R. M., & Peytcheva, E. (2008). The impact of nonresponse rates on nonresponse bias: A meta-analysis. Public Opinion Quarterly, 72(2), 167–189. https://doi.org/10.1093/poq/nfn011
Groves, R. M. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 646–675. https://doi.org/10.1093/poq/nfl033
Groves, R. M., Cialdini, R. B., & Couper, M. P. (1992). Understanding the decision to participate in a survey. Public Opinion Quarterly, 56(4), 475–495. https://doi.org/10.1086/269338
Groves, R. M., Couper, M. P., Presser, S., Singer, E., Tourangeau, R., Acosta, G. P., & Nelson, L. (2006). Experiments in producing nonresponse bias. Public Opinion Quarterly, 70(5), 720–736. https://doi.org/10.1093/poq/nfl036
Groves, R. M., Presser, S., & Dipko, S. (2004). The role of topic interest in survey participation decisions. Public Opinion Quarterly, 68(1), 2–31. https://doi.org/10.1093/poq/nfh002
Groves, R. M., Singer, E., & Corning, A. (2000). Leverage-saliency theory of survey participation: Description and an illustration. Public Opinion Quarterly, 64(3), 299–308. https://doi.org/10.1086/317990
Keusch, F. (2015). Why do people participate in Web surveys? Applying survey participation theory to Internet survey data collection. Management Review Quarterly, 65(3), 183–216. https://doi.org/10.1007/s11301-014-0111-y
Singer, E., & Ye, C. (2013). The use and effects of incentives in surveys. The Annals of the American Academy of Political and Social Science, 645(1), 112–141. https://doi.org/10.1177/0002716212458082
Singer, E. (2011). Toward a benefit-cost theory of survey participation: Evidence, further tests, and implications. Journal of Official Statistics, 27(2), 379–392.