Skip to content

Surveys · Questionnaire design

Vague Quantifiers: What Do "Often" and "Sometimes" Really Mean?

Words such as "rarely", "sometimes" and "often" are common survey answer options, but people interpret them very differently. This article explains the evidence and shows when to replace vague quantifiers with numbers.

ModQu research article · 879 words · 9 references (APA 7th edition)

Download PDF

Many survey questions offer answer options such as "never," "rarely," "sometimes," "often" and "very often." These vague quantifiers are easy to read and quick to answer, and they avoid asking respondents for exact numbers they may not know. The problem is that they do not mean the same thing to everyone. One person's "often" is another person's "sometimes," and the same person may use "often" to mean very different frequencies for different activities.

The evidence

Researchers have tested the meaning of vague quantifiers by asking people to translate them into numbers. In an early study, respondents gave strikingly different numerical values for words such as "often" and "sometimes," with wide overlaps between neighbouring terms (Hakel, 1968). Later research showed that the meaning of a quantifier depends on what it describes: "frequently" implies a much smaller number for a rare event, such as a plane crash, than for a common one (Pepper & Prytulak, 1974).

Range chart with illustrative numbers showing what different people mean by four words when describing how often they eat out per month. Rarely: 0.5 to 2 times. Sometimes: 1 to 5. Often: 3 to 12. Very often: 6 to 20. The ranges are wide and overlap.
Figure 1. The same word can mean very different frequencies to different people, and the ranges for neighbouring words overlap. Illustrative numbers by ModQu, following the pattern reported by Hakel (1968) and Bradburn and Miles (1979).

Norman Bradburn and Carrie Miles studied vague quantifiers in a national survey about feelings such as excitement and boredom. They found that the numerical meaning of terms like "very often" varied with the feeling being described and with the characteristics of the respondent (Bradburn & Miles, 1979). Nora Cate Schaeffer later showed that different demographic groups attach different meanings to the same terms, so that comparisons between groups using vague quantifiers can be misleading. Two groups that report "often" at the same rate may in fact differ in actual frequency, and vice versa (Schaeffer, 1991).

Research mapping numerical values onto quantifiers such as "quite a bit" has confirmed that the translation varies with context and between respondents (Wright et al., 1994). The same pattern appears for probability words such as "likely" and "probable": each person uses them fairly consistently, but different people mean quite different probabilities by them (Budescu & Wallsten, 1985). Reviews of how people communicate quantities suggest that these words convey a speaker's perspective, such as whether an amount is more or less than expected, as much as a quantity (Moxey & Sanford, 2000).

Why it matters

Vague quantifiers make it hard to compare groups and to estimate actual frequencies. A company that learns that 40% of customers use a service "often" cannot tell how many times a month that means, or whether heavy users in one region mean the same as heavy users in another. Changes over time are also hard to interpret, since expectations, and therefore the meaning of "often," can change.

Numeric categories are not free of problems either. Respondents use the range of numeric options as a guide to what is typical, so the options shape the answers (Schwarz et al., 1985). But at least the categories mean the same thing to everyone, so differences between groups and changes over time can be read directly. When a survey is used to track behaviour, to size a market or to compare customer segments, this comparability usually outweighs the small extra effort that numeric options ask of respondents. Pilot testing can show whether respondents find the numeric categories easy to use and whether the range of options fits the behaviour.

An example

Suppose a gym asks members, "How often do you attend classes? Never / Rarely / Sometimes / Often / Very often." A member who attends twice a week but used to attend daily may answer "sometimes," because they compare themselves with their past habits. A new member who attends twice a week may answer "often," because it is more than they have ever exercised before. Both attend at the same rate, but the survey records them differently. A version that asks "In a typical week, how many classes do you attend? None / 1 / 2 / 3 / 4 or more" removes this ambiguity and gives the gym a figure it can plan around.

The vague version is not useless. It captures how members feel about their attendance, which may predict whether they renew. If that is what the gym wants to know, the question should say so directly, for example "Do you attend classes as often as you would like?" Asking for a frequency in vague terms mixes the two purposes and serves neither well.

Practical guidance

Conclusion

Vague quantifiers are convenient, but they measure a mixture of actual frequency and personal standards. Where the purpose is to estimate how often something happens, or to compare groups, specific numbers and numeric categories give clearer and more comparable results.

References

Bradburn, N. M., & Miles, C. (1979). Vague quantifiers. Public Opinion Quarterly, 43(1), 92–101. https://doi.org/10.1086/268494

Bradburn, N. M., Sudman, S., & Wansink, B. (2004). Asking questions: The definitive guide to questionnaire design for market research, political polls, and social and health questionnaires (Rev. ed.). Jossey-Bass.

Budescu, D. V., & Wallsten, T. S. (1985). Consistency in interpretation of probabilistic phrases. Organizational Behavior and Human Decision Processes, 36(3), 391–405. https://doi.org/10.1016/0749-5978(85)90007-X

Hakel, M. D. (1968). How often is often? American Psychologist, 23(7), 533–534. https://doi.org/10.1037/h0037716

Moxey, L. M., & Sanford, A. J. (2000). Communicating quantities: A review of psycholinguistic evidence of how expressions determine perspectives. Applied Cognitive Psychology, 14(3), 237–255. https://doi.org/10.1002/(SICI)1099-0720(200005/06)14:3<237::AID-ACP641>3.0.CO;2-R

Pepper, S., & Prytulak, L. S. (1974). Sometimes frequently means seldom: Context effects in the interpretation of quantitative expressions. Journal of Research in Personality, 8(1), 95–101. https://doi.org/10.1016/0092-6566(74)90049-X

Schaeffer, N. C. (1991). Hardly ever or constantly? Group comparisons using vague quantifiers. Public Opinion Quarterly, 55(3), 395–423. https://doi.org/10.1086/269270

Schwarz, N., Hippler, H.-J., Deutsch, B., & Strack, F. (1985). Response scales: Effects of category range on reported behavior and comparative judgments. Public Opinion Quarterly, 49(3), 388–395. https://doi.org/10.1086/268936

Wright, D. B., Gaskell, G. D., & O'Muircheartaigh, C. A. (1994). How much is 'quite a bit'? Mapping between numerical values and vague quantifiers. Applied Cognitive Psychology, 8(5), 479–496. https://doi.org/10.1002/acp.2350080506