Surveys · Incentives and engagement
Survey Invitation Design: Subject Lines, Timing and Reminders
Whether people take part in a survey often depends on the first thing they see: the letter, email or text message that invites them. This article reviews the evidence on how invitations, subject lines, personalisation, timing and reminders affect response.
Before anyone answers a survey question, they must decide whether to open the invitation, read it and act on it. Most people who are invited to take part in a survey never begin. Many of these decisions are made in seconds, on the basis of the envelope, the sender's name or an email subject line. The design of the invitation and of the contacts that follow it is therefore one of the most important influences on response, and one of the cheapest to improve (Dillman et al., 2014).
This article reviews the evidence on invitation design: the content of the invitation, subject lines, personalisation, the sender, the timing of contacts and the number and form of reminders.
Why invitations matter
Theories of survey participation emphasise that the decision to respond is often made quickly, on the basis of a few cues. Groves et al. (1992) argued that people use heuristics, mental shortcuts, when deciding whether to take part: they respond to signs of authority, legitimacy, reciprocity and the sense that others are taking part. Social exchange theory, the basis of the tailored design method, holds that people respond when they trust the sponsor, perceive benefits and expect the costs to be low (Dillman et al., 2014). An invitation is the main way to communicate all of these.
Meta-analyses of mail surveys established early that contact strategy has large effects. Heberlein and Baumgartner (1978) found that the number of contacts was one of the strongest predictors of response in published mail surveys. Fox et al. (1988) and Yammarino et al. (1991) confirmed that pre-notification, follow-up contacts and sponsorship by a university or government body increased response.
The content of the invitation
A good invitation answers the questions a recipient is likely to have, briefly and in plain language:
- Who is asking? The organisation conducting the survey and, if different, the sponsor.
- Why? The purpose of the survey and how the results will be used.
- Why me? How the person was selected and why their answers matter.
- What is involved? The approximate time required and the topics covered.
- Is it safe? How answers will be kept confidential and how data will be protected.
- How do I take part? A clear link, address or telephone number, and any incentive.
- Who can I contact? A name, email address or telephone number for questions.
Longer invitations are not necessarily better. In web surveys, recipients often read only the first lines of an email, so the most important information should come first. Kaplowitz et al. (2012) experimented with the design of email invitations and found that features such as the placement of the survey link and the length of the message affected response, with shorter messages and an early link generally performing well.
Statements about burden matter. Crawford et al. (2001) found that telling people a web survey would take less time increased the number who started it, although some later broke off when the survey took longer than stated. Accurate estimates of time are both more ethical and more likely to produce completed questionnaires.
Subject lines
In email invitations, the subject line determines whether the message is opened at all. The evidence on subject lines is less consistent than for other features, partly because effects depend on the population and sponsor. Porter and Whitcomb (2005) tested subject lines in a survey of students and found modest differences: a subject line that named the sponsor or made a direct request for help did not consistently outperform a blank or generic one. Some studies find that subject lines mentioning the topic attract people interested in it, which can introduce bias, while those emphasising the request for help or the sponsor's authority produce more balanced response.
Practical advice focuses on avoiding problems. Subject lines should not resemble marketing or spam: words such as "free" or "win", excessive punctuation and capital letters increase the chance that a message is filtered or ignored. A subject line that names a recognisable sponsor and indicates a request, such as "University of X: your views on local transport", is usually a safe choice.
Personalisation
Personalised invitations address recipients by name rather than with a generic salutation such as "Dear resident". In mail surveys, personalisation has been associated with modest increases in response (Yammarino et al., 1991). In web surveys, Heerwegh (2005) found that a personal salutation increased the login rate to a web survey by several percentage points. Joinson and Reips (2007) found that personalised salutations increased response, especially when the sender had high status, though they also noted that personalisation might reduce willingness to disclose sensitive information if respondents felt less anonymous. Sauermann and Roach (2013) found in a large experiment with scientists that personalisation increased web survey response.
Personalisation is possible only when names are known and accurate. An incorrect or misspelt name can do more harm than a generic salutation.
The sender
Who sends the invitation affects whether it is trusted. Invitations from universities and government agencies generally achieve higher response than those from commercial organisations (Fox et al., 1988). In email surveys, the sender's name and address appear before the message is opened, so they function as part of the subject line. A message from a named person at a recognisable institution is more likely to be opened than one from an anonymous or unfamiliar address. Joinson and Reips (2007) found that invitations from a sender of higher status produced higher response.
Mode of contact
The way people are contacted need not match the way they respond. In address-based surveys, mailed letters are often used to invite people to respond online. Letters are more likely to be noticed and trusted than emails from unknown senders, and they can include a prepaid incentive (Millar & Dillman, 2011; Messer & Dillman, 2011). Text messages are increasingly used for invitations and reminders, particularly for mobile-friendly surveys. Mavletova and Couper (2014) compared text-message and email invitations to a mobile web survey and found that text messages produced faster responses, although overall response rates were similar.
For telephone surveys, an advance letter sent before the first call explains the purpose of the survey and increases response. A meta-analysis by de Leeuw et al. (2007) found that advance letters increased both the response rate and the cooperation rate in telephone surveys by several percentage points on average.
Pre-notification
A short pre-notice sent a few days before the main invitation tells people that a request is coming. Meta-analyses of mail surveys found that pre-notification increased response (Fox et al., 1988; Yammarino et al., 1991), and the tailored design method recommends it as the first of a series of contacts (Dillman et al., 2014). In web surveys, the effect is less consistent; a pre-notice by email can be confused with the invitation or ignored, while a pre-notice by mail followed by an email invitation can combine the trust of a letter with the convenience of a link (Keusch, 2012).
Timing
The day and time an invitation arrives can affect whether it is noticed. Evidence on the best day for email invitations is mixed and depends on the population. Faught et al. (2004) found differences in response by day of the week and time of day in a web survey of businesses, with invitations sent early in the week performing better. For general population samples, effects of timing are usually small compared with those of the number of contacts. Invitations should avoid public holidays and periods when the target group is likely to be busy, such as the end of the financial year for business surveys.
The interval between contacts matters too. Reminders sent too soon may annoy people who intended to respond; those sent too late may arrive after the invitation has been forgotten. Common practice in web surveys is to send the first reminder three to seven days after the invitation.
Reminders
Reminders are among the most effective tools for increasing response. Cook et al. (2000), in a meta-analysis of web survey response rates, found that the number of contacts was the strongest predictor of response. Each reminder typically produces a new wave of responses, smaller than the last. Figure 1 shows the pattern.
The content of reminders matters as well as their number. The tailored design method recommends that each contact differ from the previous one, adding new information or a new appeal rather than repeating the original message (Dillman et al., 2014). A first reminder might thank those who have already responded and briefly remind others; a later one might emphasise the importance of hearing from everyone or offer a different way to respond. Muñoz-Leiva et al. (2010) found that both the personalisation and the frequency of reminders affected response in a web survey. Sauermann and Roach (2013) found that varying the content of reminders and changing the time of day they were sent increased response.
There is a limit. Each additional reminder adds fewer responses, and too many contacts can irritate recipients and damage the reputation of the sponsor. Three or four contacts in total are common in web and mail surveys. Figure 2 shows a typical sequence.
Who responds to reminders
Reminders can change the composition of the sample as well as its size. People who respond only after reminders are often different from early respondents: in some surveys they are younger, busier or less interested in the topic. If so, reminders reduce nonresponse bias. In other surveys, late respondents resemble early ones, and reminders add sample size without changing estimates. Checking estimates by the contact after which people responded shows whether further reminders are likely to make a difference.
Invitations for web surveys from lists
When a survey uses a list of email addresses, such as members of an organisation or customers of a company, the invitation can draw on the existing relationship. Keusch (2012) reviewed methods for increasing response in list-based web surveys and found that pre-notification, personalisation, reminders and incentives all had positive effects, with the number of contacts and the relationship between sender and recipient especially important. Trouteaud (2004) found that the wording of the request, such as a plea for help compared with a more neutral request, affected response in a web survey.
Describing the topic and the incentive
How the invitation describes the survey's topic can affect who responds. Groves et al. (2004) found that people were more likely to take part in surveys on topics that interested them: teachers responded more to a survey about education, for example. An invitation that emphasises the topic may therefore raise response among the interested while leaving others behind, which can bias estimates related to that interest. A more general description, such as "a survey about life in your area", reduces this risk, though it must still be honest about what the questionnaire covers.
Incentives should be mentioned clearly. Where a prepaid incentive is enclosed, the letter should explain that it is a small token of thanks. Where an incentive is promised, the invitation should say exactly what it is and how it will be delivered. Incentives are among the most consistently effective features of survey requests (Edwards et al., 2002; Singer & Ye, 2013), and their effect depends partly on whether recipients notice and trust them.
Invitations to businesses and professionals
Invitations to businesses face additional barriers. Messages may be screened by assistants or filtered by corporate email systems, and the right person to answer may not be the one who receives the request. Addressing the invitation to a named person in the relevant role, explaining why that role was chosen and offering a way to forward the request to a colleague all help. Sponsorship by an industry association or a government statistical agency adds legitimacy. Timing matters more than for households: invitations should avoid periods of peak workload, such as the end of a quarter or the financial year.
Practical guidance
The evidence supports several recommendations:
- Plan a sequence of contacts, typically three to five, rather than a single invitation.
- Vary each contact, adding new information or appeals rather than repeating the same text.
- Put the essentials first: who is asking, why, how long it will take and how to respond.
- Name a credible sender and avoid wording that looks like marketing.
- Personalise where accurate names are available.
- Consider mail contact for general population surveys, even when the response mode is the web.
- Use an advance letter before telephone calls where addresses are available.
- Be honest about burden and about how data will be used.
- Test variations experimentally when the survey is large or repeated.
An example
A professional association surveys its 12,000 members by email each year. Response has fallen to 14 per cent. The association tests a new design on a random half of members: a short email from the association's president, with the survey link in the second line, followed by two reminders that each add a new reason for taking part, and a final notice stating the closing date. Response in the test group reaches 27 per cent, compared with 15 per cent in the control group, and the test group's responses include a larger share of younger members (illustrative example).
Conclusion
The invitation is the first and often the only part of a survey that most sampled people see. Clear, credible and personal invitations, sent as part of a planned sequence of varied contacts, can raise response substantially at little cost. Subject lines and timing have smaller and less consistent effects than the number and content of contacts, the identity of the sender and the choice of contact mode. Because what works varies by population, the best invitation design for a particular survey is one that has been tested with its intended recipients.
References
Cook, C., Heath, F., & Thompson, R. L. (2000). A meta-analysis of response rates in web- or internet-based surveys. Educational and Psychological Measurement, 60(6), 821–836. https://doi.org/10.1177/00131640021970934
Crawford, S. D., Couper, M. P., & Lamias, M. J. (2001). Web surveys: Perceptions of burden. Social Science Computer Review, 19(2), 146–162. https://doi.org/10.1177/089443930101900202
de Leeuw, E., Callegaro, M., Hox, J., Korendijk, E., & Lensvelt-Mulders, G. (2007). The influence of advance letters on response in telephone surveys: A meta-analysis. Public Opinion Quarterly, 71(3), 413–443. https://doi.org/10.1093/poq/nfm014
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method (4th ed.). Wiley.
Edwards, P., Roberts, I., Clarke, M., DiGuiseppi, C., Pratap, S., Wentz, R., & Kwan, I. (2002). Increasing response rates to postal questionnaires: Systematic review. BMJ, 324(7347), 1183. https://doi.org/10.1136/bmj.324.7347.1183
Faught, K. S., Whitten, D., & Green, K. W., Jr. (2004). Doing survey research on the Internet: Yes, timing does matter. Journal of Computer Information Systems, 44(3), 26–34. https://doi.org/10.1080/08874417.2004.11647579
Fox, R. J., Crask, M. R., & Kim, J. (1988). Mail survey response rate: A meta-analysis of selected techniques for inducing response. Public Opinion Quarterly, 52(4), 467–491. https://doi.org/10.1086/269125
Groves, R. M., Cialdini, R. B., & Couper, M. P. (1992). Understanding the decision to participate in a survey. Public Opinion Quarterly, 56(4), 475–495. https://doi.org/10.1086/269338
Groves, R. M., Presser, S., & Dipko, S. (2004). The role of topic interest in survey participation decisions. Public Opinion Quarterly, 68(1), 2–31. https://doi.org/10.1093/poq/nfh002
Heberlein, T. A., & Baumgartner, R. (1978). Factors affecting response rates to mailed questionnaires: A quantitative analysis of the published literature. American Sociological Review, 43(4), 447–462. https://doi.org/10.2307/2094771
Heerwegh, D. (2005). Effects of personal salutations in e-mail invitations to participate in a web survey. Public Opinion Quarterly, 69(4), 588–598. https://doi.org/10.1093/poq/nfi053
Joinson, A. N., & Reips, U.-D. (2007). Personalized salutation, power of sender and response rates to Web-based surveys. Computers in Human Behavior, 23(3), 1372–1383. https://doi.org/10.1016/j.chb.2004.12.011
Kaplowitz, M. D., Lupi, F., Couper, M. P., & Thorp, L. (2012). The effect of invitation design on web survey response rates. Social Science Computer Review, 30(3), 339–349. https://doi.org/10.1177/0894439311419084
Keusch, F. (2012). How to increase response rates in list-based web survey samples. Social Science Computer Review, 30(3), 380–388. https://doi.org/10.1177/0894439311409709
Mavletova, A., & Couper, M. P. (2014). Mobile web survey design: Scrolling versus paging, SMS versus e-mail invitations. Journal of Survey Statistics and Methodology, 2(4), 498–518. https://doi.org/10.1093/jssam/smu015
Messer, B. L., & Dillman, D. A. (2011). Surveying the general public over the internet using address-based sampling and mail contact procedures. Public Opinion Quarterly, 75(3), 429–457. https://doi.org/10.1093/poq/nfr021
Millar, M. M., & Dillman, D. A. (2011). Improving response to web and mixed-mode surveys. Public Opinion Quarterly, 75(2), 249–269. https://doi.org/10.1093/poq/nfr003
Muñoz-Leiva, F., Sánchez-Fernández, J., Montoro-Ríos, F., & Ibáñez-Zapata, J. Á. (2010). Improving the response rate and quality in Web-based surveys through the personalization and frequency of reminder mailings. Quality & Quantity, 44(5), 1037–1052. https://doi.org/10.1007/s11135-009-9256-5
Porter, S. R., & Whitcomb, M. E. (2005). E-mail subject lines and their effect on web survey viewing and response. Social Science Computer Review, 23(3), 380–387. https://doi.org/10.1177/0894439305275912
Sauermann, H., & Roach, M. (2013). Increasing web survey response rates in innovation research: An experimental study of static and dynamic contact design features. Research Policy, 42(1), 273–286. https://doi.org/10.1016/j.respol.2012.05.003
Singer, E., & Ye, C. (2013). The use and effects of incentives in surveys. The Annals of the American Academy of Political and Social Science, 645(1), 112–141. https://doi.org/10.1177/0002716212458082
Trouteaud, A. R. (2004). How you ask counts: A test of internet-related components of response rates to a web-based survey. Social Science Computer Review, 22(3), 385–392. https://doi.org/10.1177/0894439304265650
Yammarino, F. J., Skinner, S. J., & Childers, T. L. (1991). Understanding mail survey response behavior: A meta-analysis. Public Opinion Quarterly, 55(4), 613–639. https://doi.org/10.1086/269284