Intelligence · Market and competitive intelligence
Pricing Intelligence and Market Monitoring
Price is the fastest lever a business can pull and the one competitors watch most closely. This article explains how pricing intelligence is gathered, how to understand what customers actually pay and how sensitive they are to price, how automated price monitoring has changed markets, and the legal and ethical limits involved.
Of all the decisions a business makes, pricing has the most immediate effect on profit. Marn and Rosiello (1992), analysing the cost structures of a large sample of companies, calculated that for the average company a 1% improvement in price, with volume unchanged, would raise operating profit by about 11%, a larger effect than equivalent improvements in volume or costs. Price is also the variable competitors watch most closely and can change most quickly. Pricing intelligence is the systematic collection and analysis of information about prices, both one's own and competitors', and about how customers respond to them. This article explains how it is done, what it reveals and where its limits lie.
Three questions pricing intelligence answers
Pricing intelligence addresses three related questions:
- What are competitors charging? List prices, promotions, discounts, bundles and terms across products, channels and regions.
- What do we actually realise? The prices customers really pay after all discounts, rebates and allowances, which often differ greatly from list prices.
- How do customers respond to price? How sensitive demand is to price changes, how customers perceive prices and what they consider fair.
Nagle and Müller (2018) argue that effective pricing is strategic rather than reactive: it rests on understanding the value customers receive, the costs of serving them and the competitive context, rather than on matching competitors' prices or adding a margin to cost. Pricing intelligence provides the evidence for that understanding.
Monitoring competitors' prices
Sources
Competitors' prices can be gathered from many legitimate sources: published price lists and catalogues, websites and online marketplaces, retail shelves, advertising and promotions, public procurement records, quotes that customers choose to share, and industry price surveys conducted by independent parties. Fleisher and Bensoussan (2015) note that sales staff hear about competitors' prices constantly but that this information is rarely captured systematically.
Automated collection
Online commerce has transformed price monitoring. Software can collect prices from thousands of web pages daily. Cavallo and Rigobon (2016) describe the Billion Prices Project, which collected daily online prices from hundreds of retailers in many countries to produce price indices. Their work showed that online prices could track official inflation statistics closely and provide earlier and more detailed information, and in one case revealed a large gap between online price data and official statistics that were later called into question.
A natural question is whether online prices reflect prices in physical stores. Cavallo (2017), comparing online and in-store prices simultaneously at large retailers in ten countries, found that prices were identical about 72% of the time on average, with considerable variation between retailers and countries. For many multichannel retailers, online prices are therefore a reasonable guide to store prices, but not always.
Context matters
A competitor's price is meaningful only in context. The same nominal price may come with different pack sizes, quality, delivery terms, warranties or loyalty rewards. Good pricing intelligence compares like with like, recording the attributes that affect value, and tracks prices over time to distinguish permanent changes from temporary promotions.
Frequency also matters. In markets where prices change daily, a monthly check misses most of the movement; in markets where prices change once a year, daily collection wastes effort. The collection rhythm should match how often prices actually change and how quickly the business can respond.
Understanding realised prices
Many companies know their list prices well and their realised prices poorly. Marn and Rosiello (1992) introduced the price waterfall: a breakdown of how a list price is reduced by successive discounts, rebates, payment terms, promotional allowances, freight and other concessions to arrive at the pocket price the company actually keeps. Figure 1 shows a hypothetical example.
Marn and Rosiello found that pocket prices for the same product often varied widely across customers, and that the variation was frequently unrelated to customer size or value. Discounts accumulated through individual negotiations and exceptions that no one reviewed as a whole. Analysing the waterfall customer by customer reveals where value leaks, which customers are unprofitable and where discount policies are not followed. For many companies this internal intelligence is more valuable than monitoring competitors.
Understanding customer response
Price sensitivity
How much does demand change when price changes? Economists measure this as price elasticity. Bijmolt et al. (2005), in a meta-analysis of studies across many products and markets, found an average price elasticity of about −2.6, meaning that a 1% price increase was associated with a fall of about 2.6% in sales of the brand, with substantial variation by product category, stage of the product life cycle and other factors. Elasticities for individual brands are usually much larger than for categories as a whole, because customers switch between brands more readily than they stop buying a category.
Elasticity can be estimated from historical sales data, but such estimates are often confounded: prices are changed for reasons, such as promotions timed with advertising, that also affect sales. Experiments provide cleaner evidence. Kohavi et al. (2009) describe how online businesses run controlled experiments, randomly assigning customers to different versions of a website; price tests are possible in the same way, though they require care to avoid customer resentment.
Willingness to pay
Before a product is launched, or when considering a new pricing structure, survey methods can estimate willingness to pay. Green and Srinivasan (1990) review conjoint analysis, in which respondents choose between product profiles that vary in features and price, allowing analysts to estimate how much each feature is worth to customers and how demand responds to price. Conjoint studies are widely used to set prices for new products and to design product ranges.
Perception and fairness
Customers do not judge prices purely on numbers. Tversky and Kahneman (1981) showed that the way choices are framed, for example as gains or losses, changes how people evaluate them. A discount presented as a saving may be received differently from the same price presented as a surcharge avoided. Kahneman et al. (1986) found that people judge price increases as unfair when they exploit a shift in demand, such as raising the price of snow shovels after a storm, but as fair when they pass on increased costs. Perceived unfairness can provoke lasting customer anger. Pricing intelligence that tracks only competitors' numbers misses this dimension.
Dynamic and algorithmic pricing
Revenue management
Airlines and hotels pioneered revenue management: adjusting prices continuously according to demand, capacity and time remaining before departure or arrival. Talluri and van Ryzin (2004) describe the models used to set prices and allocate capacity to maximise revenue. The approach has spread to retail, entertainment, transport and many online businesses. It depends on detailed, continuous intelligence about demand and competitors' prices.
Shapiro and Varian (1999) described how information goods allow sellers to tailor prices through versioning, offering different versions to customers with different willingness to pay, and through personalised pricing based on customer data. Einav and Levin (2014) note that the growth of detailed transaction data allows much finer analysis of how prices affect behaviour.
Algorithmic pricing
Many online sellers now use algorithms that monitor competitors' prices and adjust their own automatically. Chen et al. (2016), studying sellers on Amazon's marketplace, identified hundreds using algorithmic pricing strategies, many of which tracked the lowest competitor price or the price of the product's featured offer. Algorithmic sellers changed prices far more frequently than others and were more likely to win the prominent featured position.
Algorithmic pricing raises a competition concern. Ezrachi and Stucke (2016) argue that when competitors all use algorithms that monitor and respond to each other's prices instantly, the result may be tacit coordination at higher prices, even without any agreement. Calvano et al. (2020) showed in simulations that self-learning pricing algorithms, interacting repeatedly, learned to sustain prices above competitive levels and to punish deviations, without being designed to collude or communicating with each other. Regulators are studying how existing competition law applies.
People and algorithms
Automated pricing does not remove the need for judgement. Dietvorst et al. (2015) found that people often lose trust in algorithms after seeing them make errors, even when the algorithms outperform human judgement. Pricing teams may override sound algorithmic recommendations after a few visible mistakes, or, conversely, accept recommendations in circumstances for which the algorithm was not designed. Clear rules about when to override, and monitoring of results, help.
Competitive dynamics
Pricing intelligence must anticipate competitors' reactions, not just record their current prices. Porter (1980) described how industry conditions, such as high fixed costs, excess capacity and undifferentiated products, make price competition more likely. Chen (1996) showed that firms with overlapping markets and similar resources are most likely to respond to each other's moves. Zajac and Bazerman (1991) found that managers often fail to anticipate competitors' responses, cutting prices in the belief that rivals will not follow and then finding themselves in a price war that leaves all firms worse off. A price change should be accompanied by an explicit assessment of how each major competitor is likely to respond.
Legal and ethical limits
Gathering competitors' prices from public sources is lawful. Obtaining them directly from competitors, or exchanging current or future pricing information with them, is not. In Canada, the Competition Act (1985) makes agreements between competitors to fix prices a criminal offence, and exchanges of competitively sensitive pricing information can be evidence of such agreements. Industry price benchmarks should be run by independent parties using historical, aggregated data.
Methods of collection also matter ethically. Crane (2005) argues that misrepresentation, such as posing as a customer to obtain a quote, crosses the line into improper intelligence-gathering. Requests for quotes should be made only for genuine purchasing needs, or by identifying oneself honestly.
An illustration
A distributor of building supplies believes it is losing business to a larger rival on price and asks its pricing team to recommend list price cuts. The team first builds price waterfalls for its main product lines. It finds that pocket prices for the same products vary by more than 20% across customers of similar size, mainly because of extended payment terms and freight absorbed for some customers but not others, and that several large accounts are barely profitable once these concessions are counted.
Next, the team compiles competitor prices from public sources: the rival's online catalogue, prices in public procurement awards and quotes that contractors volunteer during negotiations. The comparison shows that the rival's list prices are only slightly lower, but its delivery is faster and its online ordering easier. Win-loss interviews with contractors confirm that delivery speed matters more to many of them than small price differences.
The recommendation changes. Instead of cutting list prices, the distributor tightens its discount rules, charges for freight on small orders, and invests the recovered margin in faster delivery for its best customers. It also models how the rival would respond to a price cut and concludes that a cut would probably be matched, reducing margins for both. Over the following year, average pocket prices rise and the share of business lost to the rival falls.
Building a pricing intelligence capability
- Know your own prices. Build price waterfalls and track pocket prices by customer, product and channel.
- Monitor competitors systematically. Define which competitors, products and channels to track, collect prices consistently and record context.
- Measure customer response. Estimate elasticities, run experiments where feasible and use conjoint studies for new offers.
- Watch perceptions. Track how customers perceive price and fairness, not only what they pay.
- Anticipate reactions. Assess likely competitor responses before changing prices.
- Govern algorithms. Set rules and monitor outcomes for automated pricing, including competition law risks.
- Stay within legal and ethical limits. Use public sources, avoid exchanges with competitors and never misrepresent.
Conclusion
Pricing intelligence combines monitoring of competitors' prices, analysis of what the company actually realises and evidence about how customers respond. Research shows that small improvements in price have large effects on profit, that much value leaks through unmanaged discounts, that customers respond to framing and fairness as well as numbers, and that algorithms are changing how prices are set and how markets behave. Organisations that build a disciplined, lawful pricing intelligence capability make better pricing decisions and avoid the price wars that come from misreading competitors.
References
Bijmolt, T. H. A., van Heerde, H. J., & Pieters, R. G. M. (2005). New empirical generalizations on the determinants of price elasticity. Journal of Marketing Research, 42(2), 141–156. https://doi.org/10.1509/jmkr.42.2.141.62296
Calvano, E., Calzolari, G., Denicolò, V., & Pastorello, S. (2020). Artificial intelligence, algorithmic pricing, and collusion. American Economic Review, 110(10), 3267–3297. https://doi.org/10.1257/aer.20190623
Cavallo, A., & Rigobon, R. (2016). The Billion Prices Project: Using online prices for measurement and research. Journal of Economic Perspectives, 30(2), 151–178. https://doi.org/10.1257/jep.30.2.151
Cavallo, A. (2017). Are online and offline prices similar? Evidence from large multi-channel retailers. American Economic Review, 107(1), 283–303. https://doi.org/10.1257/aer.20160542
Chen, L., Mislove, A., & Wilson, C. (2016). An empirical analysis of algorithmic pricing on Amazon Marketplace. In Proceedings of the 25th International Conference on World Wide Web (pp. 1339–1349). https://doi.org/10.1145/2872427.2883089
Chen, M.-J. (1996). Competitor analysis and interfirm rivalry: Toward a theoretical integration. Academy of Management Review, 21(1), 100–134. https://doi.org/10.5465/amr.1996.9602161567
Competition Act, R.S.C. 1985, c. C-34 (Canada).
Crane, A. (2005). In the company of spies: When competitive intelligence gathering becomes industrial espionage. Business Horizons, 48(3), 233–240. https://doi.org/10.1016/j.bushor.2004.11.005
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015). Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General, 144(1), 114–126. https://doi.org/10.1037/xge0000033
Einav, L., & Levin, J. (2014). Economics in the age of big data. Science, 346(6210), Article 1243089. https://doi.org/10.1126/science.1243089
Ezrachi, A., & Stucke, M. E. (2016). Virtual competition: The promise and perils of the algorithm-driven economy. Harvard University Press.
Fleisher, C. S., & Bensoussan, B. E. (2015). Business and competitive analysis: Effective application of new and classic methods (2nd ed.). Pearson Education.
Green, P. E., & Srinivasan, V. (1990). Conjoint analysis in marketing: New developments with implications for research and practice. Journal of Marketing, 54(4), 3–19. https://doi.org/10.1177/002224299005400402
Kahneman, D., Knetsch, J. L., & Thaler, R. (1986). Fairness as a constraint on profit seeking: Entitlements in the market. American Economic Review, 76(4), 728–741.
Kohavi, R., Longbotham, R., Sommerfield, D., & Henne, R. M. (2009). Controlled experiments on the web: Survey and practical guide. Data Mining and Knowledge Discovery, 18(1), 140–181. https://doi.org/10.1007/s10618-008-0114-1
Marn, M. V., & Rosiello, R. L. (1992). Managing price, gaining profit. Harvard Business Review, 70(5), 84–94.
Nagle, T. T., & Müller, G. (2018). The strategy and tactics of pricing: A guide to growing more profitably (6th ed.). Routledge.
Porter, M. E. (1980). Competitive strategy: Techniques for analyzing industries and competitors. Free Press.
Shapiro, C., & Varian, H. R. (1999). Information rules: A strategic guide to the network economy. Harvard Business School Press.
Talluri, K. T., & van Ryzin, G. J. (2004). The theory and practice of revenue management. Springer.
Tversky, A., & Kahneman, D. (1981). The framing of decisions and the psychology of choice. Science, 211(4481), 453–458. https://doi.org/10.1126/science.7455683
Zajac, E. J., & Bazerman, M. H. (1991). Blind spots in industry and competitor analysis: Implications of interfirm (mis)perceptions for strategic decisions. Academy of Management Review, 16(1), 37–56. https://doi.org/10.5465/amr.1991.4278990