Surveys · Types and applications
Product Concept Testing Through Surveys
Concept tests ask potential customers to react to a description of a new product before it is built or launched. This article explains how concept tests are designed, what they measure, why stated purchase intentions need adjusting and how to read the results.
Most new products fail, and many of the failures could have been predicted from what customers said before launch (Cooper & Kleinschmidt, 1987). A concept test is a survey that shows potential customers a description of a new product, service or idea and measures their reactions. Because it takes place before money is spent on development, production and advertising, it is one of the cheapest ways to decide which ideas to pursue, which to change and which to drop (Urban & Hauser, 1993).
What a concept is
A concept is a short statement of what the product is, what it does for the customer and why they should believe it. It usually includes a headline, a description of the main benefit, the reasons to believe the claim, and often a price and a picture. Concepts range from a few lines of text to a realistic mock advertisement or a working prototype. The more finished the concept, the closer reactions are to reactions to the real product, but the more expensive each version is to produce. Online surveys now allow researchers to show images, short videos and interactive mock-ups, which brings concept tests closer to the experience of seeing a product in a shop or online (Dahan & Hauser, 2002).
Designs
Concept tests use two main designs. In a monadic test, each respondent sees only one concept, and different concepts are compared between randomly assigned groups. This mirrors how customers usually meet a single product and avoids comparisons that would not occur in the market. In a sequential monadic test, each respondent sees several concepts in rotated order and rates each one. This needs a smaller sample but lets earlier concepts influence reactions to later ones, so the order must be randomised and the first concept seen is often analysed separately. A comparative or paired design asks respondents to choose between concepts directly, which is efficient for picking a winner but says little about whether the winner would succeed on its own (Peng & Finn, 2008).
What is measured
A typical concept test asks about:
- Purchase intention, usually on a five-point scale from "definitely would buy" to "definitely would not buy".
- Uniqueness: how different the product is from what is already available.
- Relevance or need: whether the product solves a problem the respondent has.
- Believability of the claims.
- Value for money at the stated price.
- Likes and dislikes, in the respondent's own words.
Diagnostic questions explain the purchase intention score. A concept that is seen as unique but not believable needs better evidence; one that is believable but not relevant may be aimed at the wrong customers.
Why stated intentions need adjusting
Respondents usually say they are more likely to buy than they turn out to be. The gap between intention and behaviour has been studied since Juster (1966), who showed that asking for a probability of purchase predicted behaviour better than a simple yes or no. Morrison (1979) set out the reasons intentions and purchases differ: intentions change over time, and respondents' stated intentions are imprecise measures of what they actually expect to do. Morwitz et al. (2007) found that intentions predict sales better for existing products than for new ones, and better for durable goods than for everyday items.
For this reason, practitioners do not treat the share answering "definitely" or "probably" as the expected share who will buy. Instead, they apply weights to each answer, for example counting a large fraction of "definitely" answers and a much smaller fraction of "probably" answers as likely buyers. Jamieson and Bass (1989) compared several such adjustment methods and found that they improved predictions of trial, although the best weights varied between product categories. Figure 1 shows how weighting changes the picture (illustrative values).
From concept test to forecast
Concept test results feed into forecasts of sales. Pre-test-market models combine adjusted trial estimates with expected awareness, distribution and repeat purchase to forecast first-year sales. The ASSESSOR model of Silk and Urban (1978) was an early example that combined survey measures from a simulated shopping setting with models of trial and repeat. Such models depend on norms from earlier tests in the same category, which is why companies that test many concepts can interpret new results more accurately than those testing for the first time.
Practical guidance
- Define the target customer and screen respondents so that the sample matches the people the product is meant for.
- Include a benchmark, such as a concept for an existing successful product, so that scores can be interpreted against a known result.
- Show the price, because purchase intention without a price is close to meaningless for most products.
- Ask open questions to learn why respondents like or dislike the concept.
- Use enough respondents, typically at least 150 to 200 per concept in a monadic test, so that differences between concepts are larger than the margin of error.
- Treat results as relative. Concept tests are better at ranking ideas than at predicting exact sales.
An example
A food company has three ideas for a new snack bar. It runs a monadic online test with 200 target customers per concept. Concept B has the highest stated intention, 62 per cent "definitely" or "probably" would buy, but after weighting its expected trial is 24 per cent, close to concept A. Diagnostic questions show that concept B scores low on believability: respondents doubt its claim of "no added sugar" given its taste description. The company revises the claim and retests the concept before development (illustrative example).
Conclusion
Concept testing gives companies an early, inexpensive reading of how customers react to new ideas. Its value depends on careful design, a consistent presentation of concepts, diagnostic questions that explain the scores and weighting of stated intentions to reflect what customers are likely to do.
References
Cooper, R. G., & Kleinschmidt, E. J. (1987). New products: What separates winners from losers? Journal of Product Innovation Management, 4(3), 169–184. https://doi.org/10.1016/0737-6782(87)90002-6
Dahan, E., & Hauser, J. R. (2002). The virtual customer. Journal of Product Innovation Management, 19(5), 332–353. https://doi.org/10.1111/1540-5885.1950332
Jamieson, L. F., & Bass, F. M. (1989). Adjusting stated intention measures to predict trial purchase of new products: A comparison of models and methods. Journal of Marketing Research, 26(3), 336–345. https://doi.org/10.1177/002224378902600307
Juster, F. T. (1966). Consumer buying intentions and purchase probability: An experiment in survey design. Journal of the American Statistical Association, 61(315), 658–696. https://doi.org/10.1080/01621459.1966.10480897
Morrison, D. G. (1979). Purchase intentions and purchase behavior. Journal of Marketing, 43(2), 65–74. https://doi.org/10.1177/002224297904300207
Morwitz, V. G., Steckel, J. H., & Gupta, A. (2007). When do purchase intentions predict sales? International Journal of Forecasting, 23(3), 347–364. https://doi.org/10.1016/j.ijforecast.2007.05.015
Peng, L., & Finn, A. (2008). Concept testing: The state of contemporary practice. Marketing Intelligence & Planning, 26(6), 649–674. https://doi.org/10.1108/02634500810902839
Silk, A. J., & Urban, G. L. (1978). Pre-test-market evaluation of new packaged goods: A model and measurement methodology. Journal of Marketing Research, 15(2), 171–191. https://doi.org/10.1177/002224377801500201
Urban, G. L., & Hauser, J. R. (1993). Design and marketing of new products (2nd ed.). Prentice Hall.