Insights · What an insight is
Historical Examples of Transformative Insights and Their Knowledge Legacies
Nine famous insights from the history of science, medicine and statistics, from Darwin and Semmelweis to Wald and the discovery of Helicobacter pylori, examined for how each actually arose, why some were resisted, and what their patterns teach anyone who looks for insight in data today.
History is full of famous moments of insight: Archimedes in the bath, Newton under the apple tree, Kekulé dreaming of a snake. Many of these stories are simplified, and some are probably untrue. Yet when the best-documented cases are examined closely, they reveal recurring patterns in how transformative insights arise, how they are received and how they change what people know. This article examines nine well-documented cases from science, medicine and statistics. For each, it asks three questions: what was the insight, how did it actually arise, and what was its legacy? It then draws out the lessons for modern research and insight work.
1. Darwin and natural selection (1838)
In an autobiography, Charles Darwin wrote of reading, in September 1838, Thomas Malthus's essay on population "for amusement". Malthus argued that populations grow faster than their food supply, so that many individuals must die before reproducing. Darwin recalled that it "at once struck me" that under these conditions favourable variations would tend to be preserved and unfavourable ones destroyed, producing new species.
The story looks like a classic flash of insight triggered by a chance encounter. But the psychologist Howard Gruber (1974), who studied Darwin's notebooks in detail, showed that the moment was the culmination of more than a year of intense work. Darwin had already assembled most of the elements of the theory: variation, inheritance, the analogy with breeders' selection of domestic animals and the idea that species change. Reading Malthus supplied the final connecting link. The insight was sudden, but it depended on extensive preparation.
Darwin then spent twenty years gathering evidence before publishing. Darwin was finally prompted to act when Alfred Russel Wallace, working independently in the Malay Archipelago, sent Darwin an essay describing essentially the same idea. Their papers were presented together to the Linnean Society of London in 1858 (Darwin & Wallace, 1858), and On the Origin of Species followed the next year (Darwin, 1859).
Legacy. Natural selection became the organising principle of biology. The case also illustrates that major insights often occur to more than one person at about the same time, a pattern discussed below.
2. Semmelweis and handwashing (1847)
In the 1840s, Ignaz Semmelweis worked at the Vienna General Hospital, which had two maternity clinics. In the first, staffed by doctors and medical students, many more mothers died of puerperal ("childbed") fever than in the second, staffed by midwives. Semmelweis examined and rejected several explanations, such as overcrowding and the position in which women gave birth. In 1847 a colleague, Jakob Kolletschka, died after being cut with a scalpel during an autopsy, and Kolletschka's symptoms resembled those of the dying mothers. Semmelweis realised that doctors and students were coming straight from autopsies to examine women in labour, carrying what Semmelweis called "cadaverous particles" on their hands. Semmelweis introduced handwashing with a chlorinated lime solution, and deaths in the first clinic fell sharply (Best & Neuhauser, 2004).
How it arose. The insight came from a systematic comparison between two groups that differed in a known way, combined with an unexpected event that suggested a mechanism. Semmelweis had already ruled out the obvious explanations, which is the impasse that often precedes insight.
Legacy and resistance. Semmelweis's insight was largely rejected during Semmelweis's lifetime. The explanation conflicted with prevailing medical theories, implied that doctors were responsible for their patients' deaths, and was communicated poorly, without the kind of published evidence that might have persuaded colleagues. Acceptance came only after the germ theory of disease was established decades later.
3. Snow and the Broad Street pump (1854)
During a cholera outbreak in London's Soho district in 1854, the physician John Snow investigated where the victims had obtained their water. Most cholera experts at the time believed the disease spread through bad air. Snow believed it was carried by contaminated water. Snow found that deaths clustered around a public water pump on Broad Street and persuaded local officials to remove the pump's handle. Snow later published a map showing deaths as bars stacked at each address, with the pump near the centre (Snow, 1855).
How it arose. The popular version of the story, in which Snow drew the map, saw the cluster and stopped the epidemic, is a simplification. Brody, Rip, Vinten-Johansen, Paneth and Rachman (2000) showed that Snow had developed the water theory years earlier, that the epidemic was already declining when the handle was removed, and that the famous map was produced afterwards to communicate the argument. Snow's most persuasive evidence came from a separate comparison of households supplied by two water companies, one drawing water from a polluted stretch of the Thames. Koch and Denike (2009) re-examined the map and showed how Snow had addressed critics' concerns in constructing it.
Legacy. Snow's work became a founding example of epidemiology and of the use of maps and data to reveal causes. It also shows that a famous visual insight may in fact be a way of communicating an insight already reached by other means.
4. Nightingale's rose diagrams (1858)
Florence Nightingale returned from the Crimean War convinced that most soldiers who died there had been killed not by wounds but by preventable diseases spread by poor sanitation. To persuade the government and the army, Nightingale and collaborators produced statistical diagrams, now known as rose or polar-area diagrams, in which the area of each wedge showed deaths from each cause in a given month. The diagrams made the scale of preventable deaths immediately visible.
How it arose. Nightingale's insight came from direct experience combined with systematic record-keeping. The diagrams did not create the insight; they made it impossible to ignore. Brasseur (2005) analysed the diagrams as a deliberate act of visual persuasion aimed at decision-makers who would not read tables.
Legacy. Nightingale's work led to reforms in military and civilian hospital sanitation and established the idea that statistics could drive public policy. It remains a model for presenting insight to non-specialist audiences.
5. Kekulé and the benzene ring (1865)
In 1865 the chemist August Kekulé proposed that the benzene molecule is a ring of six carbon atoms, solving a puzzle that had troubled chemists. At a celebration in 1890, Kekulé said that the idea had come in a reverie of atoms dancing and a snake seizing its own tail. The story is one of the most quoted examples of insight arising from the unconscious.
How it arose. Historians have questioned the story, noting that Kekulé told it twenty-five years later and that others had proposed related structures. Rothenberg (1995) re-examined Kekulé's accounts and argued that the discovery involved a specific creative process in which opposite or separate ideas are held in mind together, rather than a simple dream. Whatever its details, the case shows how insight stories are shaped in the retelling.
Legacy. The ring structure became central to organic chemistry. The story itself became a legacy of a different kind: a model of insight as sudden inspiration that has encouraged both admiration for and suspicion of such accounts.
6. Fleming and penicillin (1928)
In 1928 Alexander Fleming returned from holiday to find that a mould had contaminated one of Fleming's culture plates of Staphylococcus bacteria, and that the bacteria around the mould had died. Many scientists would have discarded the contaminated plate. Fleming noticed the anomaly, investigated it and published the observation that the mould produced a substance that killed certain bacteria (Fleming, 1929).
How it arose. The discovery is often described as luck, but it depended on Fleming's prepared mind. Fleming had worked on antibacterial substances for years and recognised what the clear zone around the mould meant. Fleming could not purify penicillin in useful amounts, and it became a medicine only in the 1940s, after Howard Florey, Ernst Chain and colleagues at Oxford developed methods to produce and test it.
Legacy. Penicillin began the antibiotic era. The case shows that an insight is often only the beginning: turning it into value required years of further work by other people.
7. Wald and the missing bullet holes (1943)
During the Second World War, the statistician Abraham Wald worked with the Statistical Research Group at Columbia University. Military analysts had recorded where bullet holes appeared on aircraft returning from missions and proposed adding armour to the most damaged areas. Wald pointed out that the data came only from aircraft that had returned. Planes hit in the areas that showed few holes, such as the engines, were the ones that had not come back. Armour should go where the returning planes were not damaged. Mangel and Samaniego (1984) later reconstructed Wald's methods from the original memoranda.
How it arose. Wald's insight came from asking how the data had been generated. The analysts had treated the sample as if it represented all aircraft. Wald recognised that it represented only survivors.
Legacy. The case has become the standard illustration of survivorship bias, the error of drawing conclusions from cases that passed through a filter. It is directly relevant to modern research, where data on current customers, completed surveys or successful products often stand in for a wider population.
8. Marshall, Warren and stomach ulcers (1982)
For most of the twentieth century, doctors believed that peptic ulcers were caused by stress and excess stomach acid, and that bacteria could not survive in the acidic stomach. In 1982 the pathologist Robin Warren and the physician Barry Marshall in Perth, Australia, found curved bacteria in the stomachs of patients with gastritis and ulcers (Marshall & Warren, 1984). Their claim that the bacteria, later named Helicobacter pylori, caused ulcers was met with scepticism. To demonstrate the link, Marshall drank a culture of the bacteria and developed gastritis (Marshall et al., 1985).
How it arose. Warren had noticed the bacteria in biopsy samples, an observation that conflicted with the accepted view. Marshall and Warren's insight was to take the anomaly seriously rather than dismiss it as contamination.
Legacy. Treating ulcers with antibiotics transformed patient care, and Marshall and Warren received the Nobel Prize in Physiology or Medicine in 2005. Like Semmelweis, they faced resistance because their insight contradicted an established model held by experts.
9. Watson, Crick and the double helix (1953)
In early 1953, James Watson and Francis Crick in Cambridge proposed that DNA consists of two helical chains wound around each other, held together by pairs of bases: adenine with thymine, guanine with cytosine (Watson & Crick, 1953). The structure immediately suggested how genetic information could be copied, since each chain could serve as a template for the other.
How it arose. Watson's own account describes a moment of insight while moving cardboard cut-outs of the bases on a desk and seeing that the two kinds of pair had the same shape. But the moment depended on evidence produced by others, especially the X-ray diffraction images of Rosalind Franklin and Raymond Gosling at King's College London, which indicated a helix and constrained its dimensions (Franklin & Gosling, 1953). It also depended on chemical knowledge supplied by colleagues, and on many months of model-building that had produced wrong structures first.
Legacy. The double helix founded molecular biology. The case also illustrates how credit for insight can be distributed unevenly: Franklin's contribution was under-recognised for decades. In organisations, too, the person who presents an insight is not always the person whose data or observation made it possible.
Patterns across the cases
Insight follows preparation
In every case, the insight came to someone who had worked on the problem for a long time and knew the field deeply. Darwin's notebooks, Fleming's years of research on antibacterial agents and Semmelweis's systematic elimination of explanations all illustrate what Wallas (1926) called preparation. Poincaré (1910) made the same observation about Poincaré's own mathematical discoveries. Sudden insights are sudden only at the end, and research on incubation suggests why: breaks and sleep help most after substantial effort on a problem (Sio & Ormerod, 2009).
Anomalies are the trigger
Several insights began with something that did not fit: a contaminated plate, bacteria where none should survive, a death that resembled the mothers' deaths, missing bullet holes. Klein and Jarosz (2011) found that noticing contradictions is one of the most common routes to insight among modern professionals. The historical cases show the same pattern, and they also show the importance of investigating anomalies rather than discarding them.
Connections across domains
Darwin connected biology with Malthus's economics. Semmelweis connected an autopsy wound with maternal deaths. Analogical transfer of this kind is powerful but difficult, because people rarely notice that a distant situation is relevant without a prompt (Gick & Holyoak, 1980). Many historical insights depended on such prompts arriving by chance, which is why the role of luck is so often emphasised.
Many insights are multiple
Darwin and Wallace reached natural selection independently. Ogburn and Thomas (1922) listed nearly 150 cases in which inventions or discoveries were made independently by two or more people, and Merton (1961) argued that such "multiples" are the normal pattern in science. When the necessary knowledge and tools exist, several people are likely to make the same connection. Simonton (2004) argued that chance, logic, individual genius and the spirit of the times all play a part. Insight is individual, but it depends heavily on the state of shared knowledge.
New data and instruments open the way
Several insights depended on a new source of evidence. Semmelweis had mortality records for two comparable clinics; Snow had addresses of victims and the records of water companies; Nightingale had systematic counts of deaths by cause; Warren had stained biopsy samples that made the bacteria visible; Watson and Crick had X-ray diffraction images. In each case, the insight became possible when someone looked carefully at data that had not been examined in that way before. This pattern has a direct modern parallel. New data sources, such as transaction records, digital behaviour and open-ended text at scale, create opportunities for insight, but only for people who examine them with a question in mind and the knowledge to recognise what matters. Data alone did not produce any of these insights; people with prepared minds, looking at new evidence, did.
Stories simplify
The Kekulé dream, Snow's map and Darwin's reading of Malthus have all been simplified in the retelling. Insight stories tend to compress long processes into single moments and to emphasise inspiration over work. Weisberg (2015) argued that many famous insights are better explained by ordinary thinking built on extensive knowledge than by special flashes of inspiration. The simplified stories are memorable, but they give a misleading picture of how insight is actually produced.
Insight is often resisted
Semmelweis, and Marshall and Warren, faced strong resistance because their insights contradicted the established models of their professions. Kuhn (1962) described how scientific communities defend their established framework until anomalies accumulate that it cannot explain. Resistance was not simply stubbornness: the established models had explained much, and the new evidence was at first limited. But in both cases the cost of delay was measured in lives.
Communication shapes legacy
Snow's map and Nightingale's diagrams became famous because they communicated insight clearly to people who did not need to follow the technical argument. Semmelweis's poor communication contributed to the rejection of a correct insight. The legacy of an insight depends not only on its truth but on how it is presented.
Lessons for modern insight work
These cases come from science and medicine, but their lessons apply directly to research and analytics in organisations.
- Invest in preparation. Deep knowledge of the data and the domain is the precondition for insight. There are no shortcuts.
- Investigate anomalies. The unexpected finding is often the most valuable. Ask whether it is an error before discarding it, but do not assume that it is.
- Ask how the data were generated. Wald's insight came from questioning the sample. Modern data, such as reviews from customers who chose to write them, surveys of people who chose to respond, or records of customers who stayed, often have hidden filters.
- Compare groups that differ in a known way. Semmelweis's two clinics and Snow's two water companies were natural experiments. Looking for such comparisons in existing data can reveal causes.
- Bring in other domains. Many insights came from connecting a problem with knowledge from elsewhere. Diverse teams and wide reading increase the chance of such connections.
- Expect resistance to insights that challenge established views. Prepare stronger evidence and clearer communication for these than for insights that confirm existing beliefs.
- Communicate visually and simply. Nightingale and Snow showed that a well-designed chart can turn an insight into action.
- Remember that insight is the beginning. Fleming's discovery needed a decade of further work by others. Organisations should plan for the work needed to develop an insight, not only to find it.
Key takeaways
The best-documented historical insights show consistent patterns. They follow long preparation, are often triggered by anomalies, frequently connect knowledge from different domains and are often reached independently by more than one person. Popular stories compress them into single inspired moments, but the real processes were longer and more collaborative. Transformative insights that contradict established models are frequently resisted, and their legacy depends on the quality of the evidence and how clearly it is communicated. For modern research teams, these cases are a reminder that insight comes from deep knowledge, careful attention to what does not fit, scepticism about how data were produced and persuasive communication.
References
Best, M., & Neuhauser, D. (2004). Ignaz Semmelweis and the birth of infection control. Quality and Safety in Health Care, 13(3), 233–234. https://doi.org/10.1136/qshc.2004.010918
Brasseur, L. (2005). Florence Nightingale's visual rhetoric in the rose diagrams. Technical Communication Quarterly, 14(2), 161–182. https://doi.org/10.1207/s15427625tcq1402_3
Brody, H., Rip, M. R., Vinten-Johansen, P., Paneth, N., & Rachman, S. (2000). Map-making and myth-making in Broad Street: The London cholera epidemic, 1854. The Lancet, 356(9223), 64–68. https://doi.org/10.1016/S0140-6736(00)02442-9
Darwin, C. (1859). On the origin of species by means of natural selection. John Murray.
Darwin, C., & Wallace, A. (1858). On the tendency of species to form varieties; and on the perpetuation of varieties and species by natural means of selection. Journal of the Proceedings of the Linnean Society of London. Zoology, 3(9), 45–62. https://doi.org/10.1111/j.1096-3642.1858.tb02500.x
Fleming, A. (1929). On the antibacterial action of cultures of a penicillium, with special reference to their use in the isolation of B. influenzæ. British Journal of Experimental Pathology, 10(3), 226–236.
Franklin, R. E., & Gosling, R. G. (1953). Molecular configuration in sodium thymonucleate. Nature, 171(4356), 740–741. https://doi.org/10.1038/171740a0
Gick, M. L., & Holyoak, K. J. (1980). Analogical problem solving. Cognitive Psychology, 12(3), 306–355. https://doi.org/10.1016/0010-0285(80)90013-4
Gruber, H. E. (1974). Darwin on man: A psychological study of scientific creativity. Dutton.
Klein, G., & Jarosz, A. (2011). A naturalistic study of insight. Journal of Cognitive Engineering and Decision Making, 5(4), 335–351. https://doi.org/10.1177/1555343411427013
Koch, T., & Denike, K. (2009). Crediting his critics' concerns: Remaking John Snow's map of Broad Street cholera, 1854. Social Science & Medicine, 69(8), 1246–1251. https://doi.org/10.1016/j.socscimed.2009.07.046
Kuhn, T. S. (1962). The structure of scientific revolutions. University of Chicago Press.
Mangel, M., & Samaniego, F. J. (1984). Abraham Wald's work on aircraft survivability. Journal of the American Statistical Association, 79(386), 259–267. https://doi.org/10.1080/01621459.1984.10478038
Marshall, B. J., & Warren, J. R. (1984). Unidentified curved bacilli in the stomach of patients with gastritis and peptic ulceration. The Lancet, 323(8390), 1311–1315. https://doi.org/10.1016/S0140-6736(84)91816-6
Marshall, B. J., Armstrong, J. A., McGechie, D. B., & Glancy, R. J. (1985). Attempt to fulfil Koch's postulates for pyloric Campylobacter. Medical Journal of Australia, 142(8), 436–439. https://doi.org/10.5694/j.1326-5377.1985.tb113443.x
Merton, R. K. (1961). Singletons and multiples in scientific discovery: A chapter in the sociology of science. Proceedings of the American Philosophical Society, 105(5), 470–486.
Ogburn, W. F., & Thomas, D. (1922). Are inventions inevitable? A note on social evolution. Political Science Quarterly, 37(1), 83–98. https://doi.org/10.2307/2142320
Poincaré, H. (1910). Mathematical creation. The Monist, 20(3), 321–335. https://doi.org/10.5840/monist19102037
Rothenberg, A. (1995). Creative cognitive processes in Kekulé's discovery of the structure of the benzene molecule. The American Journal of Psychology, 108(3), 419–438. https://doi.org/10.2307/1422898
Simonton, D. K. (2004). Creativity in science: Chance, logic, genius, and zeitgeist. Cambridge University Press.
Sio, U. N., & Ormerod, T. C. (2009). Does incubation enhance problem solving? A meta-analytic review. Psychological Bulletin, 135(1), 94–120. https://doi.org/10.1037/a0014212
Snow, J. (1855). On the mode of communication of cholera (2nd ed.). John Churchill.
Wallas, G. (1926). The art of thought. Harcourt, Brace.
Watson, J. D., & Crick, F. H. C. (1953). Molecular structure of nucleic acids: A structure for deoxyribose nucleic acid. Nature, 171(4356), 737–738. https://doi.org/10.1038/171737a0
Weisberg, R. W. (2015). Toward an integrated theory of insight in problem solving. Thinking & Reasoning, 21(1), 5–39. https://doi.org/10.1080/13546783.2014.886625