Surveys · Questionnaire design
Designing Demographic Questions That Are Inclusive and Accurate
Demographic questions about age, sex and gender, race and ethnicity, education and income look routine but are among the hardest to get right. This article explains how to design them so that they are accurate, comparable and respectful of the people who answer them.
Almost every survey ends with a set of demographic questions: age, sex or gender, ethnicity, education, employment, income, household size and location. These questions are often treated as routine and copied from one survey to the next with little thought. Yet they are essential. They are used to describe who responded, to compare groups, to check the sample against the population and to weight the results. Errors in demographic questions therefore spread into every other finding.
Demographic questions are also among the most sensitive. They ask people to place themselves in categories that relate to identity, status and sometimes disadvantage. A question that offers no category that fits, or uses outdated or offensive terms, can cause respondents to skip the question, give a false answer or abandon the survey. This article explains the principles of good demographic question design, discusses the most challenging topics and offers practical guidance.
Principles
Ask only what is needed
Each demographic question should have a clear purpose: describing the sample, comparing groups, weighting or checking representativeness. Questions that serve no analytic purpose add burden and raise privacy concerns without benefit. Guidance on reporting research samples recommends that researchers think about why each characteristic matters for their study before including it (Hughes et al., 2016).
Use recognised standards where possible
Using the same categories as national statistical agencies has two advantages. It allows survey results to be compared with census and other benchmark data, which is essential for weighting, and the standard categories have usually been tested extensively. Where standards exist, departing from them should be a deliberate choice with a reason.
Let respondents describe themselves
Fixed categories rarely fit everyone. Write-in options, such as "another category, please specify," allow respondents who do not see themselves in the list to answer accurately. Where people can belong to more than one category, as with ethnic or racial identity, allowing multiple answers is more accurate than forcing a single choice.
Explain and protect
Respondents are more willing to answer when they understand why a question is asked. A short explanation ("We ask this to check that our survey includes people from all backgrounds") can reduce nonresponse. Sensitive items should normally include "prefer not to say," and the data should be stored and reported in ways that protect confidentiality (Tourangeau & Yan, 2007).
Place them carefully
Demographic questions are usually asked at the end of a questionnaire, after respondents have engaged with the main topic and built some trust (Dillman et al., 2014). The exception is when they are needed early, for example to screen for eligibility or to manage quotas.
Challenging topics
Sex and gender
For many years, surveys asked a single question about "sex" with two options. This approach does not distinguish between sex assigned at birth and current gender identity, and it gives transgender and non-binary respondents no accurate answer. Reviews of social surveys have argued that adding new categories to a single question is not enough, because sex and gender are distinct concepts that need to be measured separately (Westbrook & Saperstein, 2015).
A widely recommended solution is the two-step approach: one question asks about sex assigned at birth and a second asks about current gender identity (Tate et al., 2013). Testing in population surveys found that this format was understood by both cisgender and transgender respondents and allowed transgender respondents to be identified accurately (Bauer et al., 2017). A major review by the United States National Academies of Sciences, Engineering, and Medicine recommended this two-step approach for use in surveys and administrative data, with careful attention to wording and to privacy (National Academies of Sciences, Engineering, and Medicine [NASEM], 2022).
For many business surveys, only one of the two concepts is needed. A customer survey that wants to report results by gender can ask a single question about current gender with inclusive options, while a health survey may need both. The key is to decide which concept is relevant and to measure it directly.
Race and ethnicity
Questions about race and ethnicity are shaped by history, law and politics, and the categories used differ between countries (Kertzer & Arel, 2002). In the United States, for example, census race categories have changed repeatedly over two centuries, reflecting changing ideas about identity as much as changes in the population (Prewitt, 2013). In 2024, the federal standards were revised to combine race and ethnicity into a single question, to add a Middle Eastern or North African category and to encourage the collection of more detailed categories (Office of Management and Budget, 2024).
Several lessons apply widely. Respondents should be able to choose more than one category. Detailed categories, or write-in options, give better information than broad groups, which can hide large differences. And identification can change: a study that followed the same individuals over many years found that a substantial number reported different racial identities at different times, and that changes were associated with life events such as unemployment and incarceration (Saperstein & Penner, 2012). Surveys should therefore treat race and ethnicity as self-identification at a point in time.
Sexual orientation
Questions about sexual orientation are increasingly included in social and health surveys. The same principles apply: a clear question, categories that respondents recognise, a write-in option and "prefer not to say." The National Academies review recommended a single question asking how respondents describe themselves, with options such as lesbian or gay, straight, bisexual and a write-in, and stressed that such questions should be asked only where the information serves a clear purpose and can be protected (NASEM, 2022). Self-administered modes are preferable to interviewer administration, because respondents are more willing to disclose sensitive information when no interviewer is present (Tourangeau & Yan, 2007).
Household, employment and language
Other common demographic items have their own pitfalls. Household size depends on who counts as a household member, and respondents differ on whether to include students living away, lodgers or partners who stay part of the week. A short definition helps. Employment questions must allow for people with more than one job, the self-employed, students who work and people on leave. A single question with options such as "employed," "self-employed," "student" and "retired" forces many respondents to choose between categories that both apply; a question about main activity, or separate questions about work and study, avoids this. Questions about language should distinguish between the language first learned, the language spoken most often at home and the language in which the respondent is most comfortable answering, since each serves a different purpose.
Education
Education seems straightforward, but categories differ between countries and change over time. A question that asks for "highest level of education completed" must use categories that map onto the national system and onto international standards if comparisons are needed. Research comparing education measures across European countries has shown that categories that look equivalent may measure different things in different countries, which can distort comparisons (Schneider, 2010; Hoffmeyer-Zlotnik & Wolf, 2003). For international surveys, translating education categories is a matter of matching qualifications, not just words (Harkness et al., 2003).
Income
Income is one of the most sensitive demographic questions and has among the highest rates of item nonresponse (Moore et al., 2000). It is also hard to report accurately, because people may not know their household's total income, may confuse gross and net income or may forget irregular sources. Good income questions specify the unit (person or household), the period (last year or last month), whether the amount is before or after tax and which sources to include. Offering income ranges rather than asking for an exact amount reduces nonresponse, and asking those who decline a series of yes/no questions about whether income is above or below certain amounts can recover further information (Juster & Smith, 1997).
Age
Age is usually best asked as date of birth or exact age in years, rather than in categories. Exact values can be grouped later in any way needed, while categories cannot be regrouped. Where privacy concerns are high, year of birth is a good compromise.
Marital and family status
Questions about marital status were once simple lists of married, single, divorced and widowed. Today many couples live together without marrying, some people are separated but not divorced and same-sex couples may be married, in civil partnerships or cohabiting. A question about legal marital status and a separate question about whether the respondent lives with a partner capture these situations more accurately than a single list. Questions about children should specify whether they mean children of any age or only those living in the household, and whether stepchildren and foster children count. As with other demographics, the right version depends on what the analysis needs.
Demographic questions in business surveys
Business surveys often borrow demographic questions without considering whether they are needed. A short customer feedback survey may not need any demographics beyond what the company already knows from its records. Where demographics are needed, for example to check that the sample matches the customer base or to compare segments, they should match the categories used in the comparison data. A company that weights its survey to census figures must use census-compatible categories for age, region and other weighting variables, or the weighting cannot be done properly.
Inclusive design also affects response. Respondents who meet an outdated or offensive question near the end of a survey may abandon it, and those who do complete it may trust the organisation less. Testing demographic questions with a varied group of customers or employees before launch costs little and avoids both problems. Organisations that survey internationally should check that each demographic question makes sense in every country and language, rather than translating categories designed for one country.
Testing demographic questions
Demographic questions should be tested like any other. Cognitive interviews with people from a range of backgrounds, especially those whose identities may not fit standard categories, are the best way to find missing options, confusing terms and wording that respondents find offensive (Fowler, 1995). Pilot data show which questions are frequently skipped and whether write-in answers reveal categories that should be added.
Conclusion
Demographic questions are the foundation of survey analysis, and they deserve as much care as any other part of the questionnaire. Good demographic questions are purposeful, use recognised standards where possible, allow respondents to describe themselves accurately, explain why they are asked and protect the answers. The hardest topics, including sex and gender, race and ethnicity, education and income, benefit from current guidance developed by statistical agencies and researchers. Designing these questions well makes results more accurate and comparable and shows respect for the people who provide the data (Krosnick & Presser, 2010; Groves et al., 2009; Bradburn et al., 2004).
References
Bauer, G. R., Braimoh, J., Scheim, A. I., & Dharma, C. (2017). Transgender-inclusive measures of sex/gender for population surveys: Mixed-methods evaluation and recommendations. PLOS ONE, 12(5), Article e0178043. https://doi.org/10.1371/journal.pone.0178043
Bradburn, N. M., Sudman, S., & Wansink, B. (2004). Asking questions: The definitive guide to questionnaire design for market research, political polls, and social and health questionnaires (Rev. ed.). Jossey-Bass.
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method (4th ed.). Wiley.
Fowler, F. J., Jr. (1995). Improving survey questions: Design and evaluation. SAGE.
Groves, R. M., Fowler, F. J., Jr., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2009). Survey methodology (2nd ed.). Wiley.
Harkness, J. A., van de Vijver, F. J. R., & Mohler, P. P. (Eds.). (2003). Cross-cultural survey methods. Wiley.
Hoffmeyer-Zlotnik, J. H. P., & Wolf, C. (Eds.). (2003). Advances in cross-national comparison: A European working book for demographic and socio-economic variables. Kluwer Academic/Plenum. https://doi.org/10.1007/978-1-4419-9186-7
Hughes, J. L., Camden, A. A., & Yangchen, T. (2016). Rethinking and updating demographic questions: Guidance to improve descriptions of research samples. Psi Chi Journal of Psychological Research, 21(3), 138–151. https://doi.org/10.24839/2164-8204.JN21.3.138
Juster, F. T., & Smith, J. P. (1997). Improving the quality of economic data: Lessons from the HRS and AHEAD. Journal of the American Statistical Association, 92(440), 1268–1278. https://doi.org/10.1080/01621459.1997.10473648
Kertzer, D. I., & Arel, D. (Eds.). (2002). Census and identity: The politics of race, ethnicity, and language in national censuses. Cambridge University Press.
Krosnick, J. A., & Presser, S. (2010). Question and questionnaire design. In P. V. Marsden & J. D. Wright (Eds.), Handbook of survey research (2nd ed., pp. 263–313). Emerald.
Moore, J. C., Stinson, L. L., & Welniak, E. J., Jr. (2000). Income measurement error in surveys: A review. Journal of Official Statistics, 16(4), 331–361.
National Academies of Sciences, Engineering, and Medicine. (2022). Measuring sex, gender identity, and sexual orientation. The National Academies Press. https://doi.org/10.17226/26424
Office of Management and Budget. (2024). Revisions to OMB's Statistical Policy Directive No. 15: Standards for maintaining, collecting, and presenting federal data on race and ethnicity. Federal Register, 89(62). https://www.federalregister.gov/documents/2024/03/29/2024-06469/revisions-to-ombs-statistical-policy-directive-no-15-standards-for-maintaining-collecting-and
Prewitt, K. (2013). What is your race? The census and our flawed efforts to classify Americans. Princeton University Press.
Saperstein, A., & Penner, A. M. (2012). Racial fluidity and inequality in the United States. American Journal of Sociology, 118(3), 676–727. https://doi.org/10.1086/667722
Schneider, S. L. (2010). Nominal comparability is not enough: (In-)equivalence of construct validity of cross-national measures of educational attainment in the European Social Survey. Research in Social Stratification and Mobility, 28(3), 343–357. https://doi.org/10.1016/j.rssm.2010.03.001
Tate, C. C., Ledbetter, J. N., & Youssef, C. P. (2013). A two-question method for assessing gender categories in the social and medical sciences. Journal of Sex Research, 50(8), 767–776. https://doi.org/10.1080/00224499.2012.690110
Tourangeau, R., & Yan, T. (2007). Sensitive questions in surveys. Psychological Bulletin, 133(5), 859–883. https://doi.org/10.1037/0033-2909.133.5.859
Westbrook, L., & Saperstein, A. (2015). New categories are not enough: Rethinking the measurement of sex and gender in social surveys. Gender & Society, 29(4), 534–560. https://doi.org/10.1177/0891243215584758