Investigation · Journalism and public investigation
Fact-Checking: Methods and Effectiveness
Fact-checking examines public claims and publishes verdicts on their accuracy. This article explains how fact-checking developed, how fact-checkers choose and test claims, how their ratings work, what research shows about whether corrections change beliefs and behaviour, and the debates about the practice.
When a politician says that crime has doubled, that a policy will create a million jobs, or that a rival voted to cut pensions, how can the public know whether it is true? Fact-checking is a form of journalism devoted to answering that question. Fact-checkers select claims made by public figures, investigate the evidence and publish a reasoned verdict, often with a rating such as "true", "misleading" or "false". Since the early 2000s, fact-checking has grown from a few specialist projects into an international movement with hundreds of organisations, and its methods have been adopted by newsrooms, platforms and researchers.
Fact-checking is a small-scale investigation. Each check defines a question, gathers evidence, tests it and reports a conclusion with stated confidence. This article explains how the practice developed, how fact-checks are carried out, what the research shows about their effects, and the criticisms and limits that fact-checkers face.
Two meanings of fact-checking
The term has two meanings. The older one refers to the internal checking that some magazines and publishers carry out before publication, in which dedicated staff verify every name, date, quotation and factual statement in an article. This form of verification aims to prevent errors from being published. Shapiro et al. (2013) found, in interviews with journalists, that verification in ordinary newsrooms is often less systematic than in these dedicated departments, and depends heavily on individual judgement and time pressure. Silverman (2007) documented how errors that slip through such checks can spread and persist, and argued for more rigorous verification and transparent correction.
The newer meaning, which is the subject of most of this article, refers to external or political fact-checking: the public assessment of claims already made by politicians, officials, campaigners and others. Its purpose is not to prevent errors in the fact-checker's own publication but to hold public speakers to account for what they say and to inform citizens.
The rise of political fact-checking
Graves (2016), in a detailed study of the American fact-checking movement, traces its growth to the early 2000s, when organisations such as FactCheck.org, PolitiFact and The Washington Post's Fact Checker were founded. He argues that they emerged partly from frustration with "he said, she said" reporting, in which journalists quoted opposing claims without telling readers which was correct. Fact-checkers sought to restore journalism's role in establishing facts in public debate.
The movement spread quickly. Graves and Cherubini (2016) described the rise of fact-checking organisations across Europe, noting that they took varied forms: some based in newsrooms, others independent non-profits, and some linked to civil society or academic institutions. They found that European fact-checkers often saw their work as part of a broader reform of political journalism and public debate, and many placed more emphasis on public engagement and on influencing politicians than on strict newsroom neutrality.
In 2015, the International Fact-Checking Network (IFCN) was founded at the Poynter Institute to bring together fact-checkers from many countries. Its code of principles commits signatories to non-partisanship and fairness, transparency of sources, transparency of funding and organisation, transparency of methods, and open and honest corrections (International Fact-Checking Network [IFCN], 2016). Signatories are assessed against the code, and adherence became a condition for some partnerships with technology platforms.
How a fact-check is done
Although each organisation has its own procedures, most fact-checks follow similar steps.
Selecting claims. Fact-checkers cannot check everything, so they choose. Common criteria are that the claim is a statement of fact rather than opinion or prediction, that it is significant, that it has been widely heard or shared, and that it can be checked with available evidence. Claims about the future, such as the effect a policy will have, can be checked only in part, for example by examining the forecasts on which they rest. The selection process is itself a source of potential bias, because the choice of which claims and which speakers to check affects the overall picture that fact-checking presents.
Defining the question. A claim must be understood precisely before it can be tested. What exactly did the speaker say, and what would an ordinary listener take it to mean? A statement that "unemployment has fallen" may be literally true for one measure and period but misleading if another measure or period gives a different picture. Fact-checkers record the full quotation and context, often including video or transcripts.
Contacting the speaker. Good practice is to ask the person or organisation that made the claim for their source. This often reveals the figure or study on which the claim was based, and the point at which it was misread, exaggerated or taken out of context. It also gives the speaker a fair chance to explain.
Gathering and weighing evidence. Fact-checkers use official statistics, government documents, academic research, court records and interviews with independent experts. Graves (2016) describes how fact-checkers often rely on authoritative institutional sources, such as statistical agencies and budget offices, while recognising that these can also be contested. Where evidence is mixed or uncertain, a good fact-check explains the uncertainty rather than forcing a simple verdict.
Reaching a verdict and publishing. Many fact-checkers use rating scales, while others publish verdicts in words only. Fact-checks normally show their sources with links, explain their reasoning step by step and set out the speaker's response. If new evidence emerges, the fact-check is updated and the correction is noted.
Rating scales
Rating scales are one of the most visible and debated features of fact-checking. They give readers a quick summary and allow statements to be compared and counted, but they compress complex judgements into a few categories.
Researchers have questioned how consistent such ratings are. Uscinski and Butler (2013) argued that fact-checking often involves judgements, about which claims to select, how to interpret them and how to treat predictions and causal claims, that cannot be settled by facts alone, and that ratings can give a misleading appearance of objectivity. Amazeen (2015) responded by showing that different fact-checkers' verdicts on the same claims in political advertising agreed to a high degree, suggesting that the practice was more reliable than critics claimed. Lim (2018), however, found that two major American fact-checkers rarely checked the same statements, and that when they did, their ratings agreed less often for claims in the middle of the scale. Marietta et al. (2015) found that fact-checkers sometimes reached different conclusions on politically disputed issues such as climate change and national debt.
These debates suggest that verdicts on clear-cut claims are generally consistent, while verdicts on ambiguous or complex claims depend more on judgement. Transparent reasoning, which lets readers see how a verdict was reached and disagree with it if they wish, matters more than the rating itself.
Does fact-checking work?
The central question is whether fact-checks change what people believe and how they behave. Research has examined effects on beliefs, on attitudes towards politicians and on the behaviour of politicians themselves.
Effects on beliefs. Early research raised the possibility of a "backfire effect", in which corrections strengthened false beliefs among people whose political identity was threatened. Nyhan and Reifler (2010) reported such effects in some of their experiments on political misperceptions. However, later and larger studies found that backfire is rare. Wood and Porter (2019) tested corrections of many statements by politicians of both parties with more than 10,000 participants and found that corrections generally moved beliefs in the accurate direction, even among those who supported the politician. Porter and Wood (2021) conducted simultaneous experiments in Argentina, Nigeria, South Africa and the United Kingdom and found that fact-checks reduced belief in false claims in all four countries, with effects that were still detectable, though smaller, about two weeks later.
Walter et al. (2020) conducted a meta-analysis of experimental studies of fact-checking and found that fact-checks had a significant positive effect on the accuracy of political beliefs overall. Effects were weaker when fact-checks conflicted with people's existing beliefs and ideology, and stronger when they addressed claims about campaigns rather than about deeply held views. They also found that fact-checks using rating scales alone were less effective than those that explained the facts.
Why corrections are only partly effective. Psychological research helps explain the limits. Lewandowsky et al. (2012) reviewed evidence on the "continued influence effect", in which people continue to rely on misinformation even after it has been corrected and they remember the correction. They recommended providing an alternative explanation that fills the gap left by the false information, repeating the correct information, and warning people before they encounter misinformation. Ecker et al. (2022) updated this review and concluded that corrections do reduce misbelief, but rarely eliminate it, and that their effects fade over time unless they are repeated.
Effects on attitudes and votes. Changing beliefs about facts does not always change opinions about the people who made the claims. Nyhan et al. (2020) found that fact-checks of claims by Donald Trump during the 2016 campaign improved the accuracy of supporters' beliefs but did not reduce their support for him. Fridkin et al. (2015) found that fact-checks of negative political advertisements affected how accurate, useful and fair viewers judged the advertisements to be. These findings suggest that factual corrections and political evaluations are related but separate.
Effects on politicians. Fact-checking may matter most through its effects on the people being checked. Nyhan and Reifler (2015) conducted a field experiment in which they sent letters to United States state legislators reminding them of the risks to their reputation of making questionable statements that might be fact-checked. Legislators who received the letters were less likely to have their accuracy publicly questioned in the following months. This suggests that the prospect of being fact-checked can deter false claims.
Effects of labels on social media. Platforms have used fact-checks to label or reduce the distribution of false content. Clayton et al. (2020) found that specific "rated false" tags on false headlines reduced belief in them more than general warnings about misinformation. But Pennycook et al. (2020) identified an "implied truth effect": when some false headlines carried warnings, people judged unlabelled false headlines to be more accurate, apparently assuming that anything not labelled had been checked. Because fact-checkers can check only a small fraction of content, this effect is a real limitation of labelling strategies.
Nieminen and Rapeli (2019), reviewing the literature, conclude that fact-checking can reduce misperceptions, but that its effects are usually modest, vary with people's political views, and depend on how and when corrections are presented.
Beyond politics
Although political fact-checking is the best-known form, the same methods are applied to many other kinds of claims. Health claims are a major focus: false or misleading statements about vaccines, cancer treatments, diets and supplements can cause direct harm, and during the COVID-19 pandemic fact-checkers around the world worked on claims about the virus, its origins and treatments. Scientific claims in news reports, such as headlines that exaggerate what a single study found, are another target, and checking them usually involves reading the original research and asking independent scientists to assess it.
Commercial claims can be checked in the same way. Advertisements that state a product is "clinically proven", that a service is the "cheapest" or that a company is "carbon neutral" can be tested against the evidence behind them. For businesses, the lesson is that public claims are increasingly likely to be examined, and that claims supported by clear, published evidence are more defensible than those that rely on selective figures.
Prevention as well as cure
Because corrections are only partly effective, researchers and fact-checkers have explored ways to reduce the effect of misinformation before people encounter it, an approach often called "prebunking". Lewandowsky et al. (2012) found that warning people in advance that they might be exposed to misleading information made them less likely to rely on it. Roozenbeek and van der Linden (2019) developed an online game in which players take the role of someone creating fake news, learning common techniques such as impersonation, emotional language and conspiracy framing. Players became better at recognising misinformation that used these techniques. Such approaches complement fact-checking rather than replacing it: prebunking teaches people to recognise general tactics, while fact-checking addresses specific claims.
Criticisms and challenges
Fact-checking faces several criticisms. Some argue that it is biased, either because fact-checkers' own political views influence which claims they check and how they rate them, or because one side makes more false claims and therefore receives more negative ratings, which its supporters then see as bias. Transparency about selection and methods, and attention to balance in what is checked, are the main responses.
Others argue that fact-checking cannot keep pace with the volume and speed of false claims, especially online. False information often spreads faster than corrections: Vosoughi et al. (2018) found that false news on Twitter spread farther and faster than true news. A fact-check published a day later may reach only a fraction of the people who saw the original claim.
A further challenge is that much misleading communication does not consist of checkable false claims at all. Selective statistics, insinuations, misleading framing and true statements arranged to create a false impression are harder to rate. Fact-checkers increasingly publish explanatory pieces on such topics rather than simple verdicts.
Finally, fact-checking organisations face financial pressures. Many rely on philanthropic funding or on contracts with technology platforms, and changes in platform policies can threaten their sustainability. Kovach and Rosenstiel (2014) argue that verification is the essence of journalism, which suggests that fact-checking should be a responsibility of all newsrooms, not only specialist ones.
Automation
The scale of the problem has encouraged research on automated fact-checking. Systems such as ClaimBuster identify statements in speeches and debates that are likely to contain checkable factual claims, helping human fact-checkers decide what to examine (Hassan et al., 2017). Researchers have built datasets to train and test systems that retrieve evidence and assess claims, such as the FEVER dataset based on Wikipedia (Thorne et al., 2018). Guo et al. (2022) review this research and conclude that automated tools are useful for specific tasks, such as detecting claims worth checking, finding previously checked claims and retrieving evidence, but that fully automated verification of real-world claims remains unreliable. Human judgement remains essential, especially for claims that depend on context or contested evidence.
A short example
The following hypothetical case illustrates the method. During a debate, a candidate for mayor says: "Under the current mayor, violent crime in our city has doubled." The claim is widely shared on social media.
A fact-checker selects it because it is specific, significant and widely heard. The first step is to define it: the candidate is referring to police-recorded violent crime, and "under the current mayor" means since she took office four years ago. The fact-checker asks the candidate's campaign for its source and is directed to a police data release.
The police figures show that recorded violent offences rose from 2,100 to 4,050 over the four years, nearly doubling. However, the police's own notes explain that recording rules changed in the second year, so that some offences previously classed as minor assaults are now counted as violent crime. An independent criminologist explains that this change accounts for much of the rise. A separate victimisation survey, which is not affected by recording practice, shows a smaller increase of about 15 per cent over the same period.
The fact-checker rates the claim "half true": the recorded figure did nearly double, but the claim leaves out the recording change and gives a misleading impression of how much crime itself increased. The published fact-check includes the police data, the explanation of the recording change, the survey figures, the criminologist's comments and the campaign's response. When the police publish revised figures that adjust for the recording change, the fact-check is updated.
Conclusion
Fact-checking applies investigative methods to public claims: selecting checkable statements, defining them precisely, seeking the speaker's evidence, testing it against authoritative sources and publishing a transparent verdict. Research shows that fact-checks generally improve the accuracy of people's beliefs, though effects are modest, fade over time and do not necessarily change political attitudes. The prospect of being checked may also deter false claims by politicians. The practice faces real limits, including selection bias, inconsistent ratings on complex claims, the speed of online misinformation and the difficulty of checking misleading but technically true statements. Its greatest strength is transparency: by showing its evidence and reasoning, a good fact-check invites the public not only to accept a verdict but to understand and test it.
References
Amazeen, M. A. (2015). Revisiting the epistemology of fact-checking. Critical Review, 27(1), 1–22. https://doi.org/10.1080/08913811.2014.993890
Clayton, K., Blair, S., Busam, J. A., Forstner, S., Glance, J., Green, G., Kawata, A., Kovvuri, A., Martin, J., Morgan, E., Sandhu, M., Sang, R., Scholz-Bright, R., Welch, A. T., Wolff, A. G., Zhou, A., & Nyhan, B. (2020). Real solutions for fake news? Measuring the effectiveness of general warnings and fact-check tags in reducing belief in false stories on social media. Political Behavior, 42(4), 1073–1095. https://doi.org/10.1007/s11109-019-09533-0
Ecker, U. K. H., Lewandowsky, S., Cook, J., Schmid, P., Fazio, L. K., Brashier, N., Kendeou, P., Vraga, E. K., & Amazeen, M. A. (2022). The psychological drivers of misinformation belief and its resistance to correction. Nature Reviews Psychology, 1(1), 13–29. https://doi.org/10.1038/s44159-021-00006-y
Fridkin, K., Kenney, P. J., & Wintersieck, A. (2015). Liar, liar, pants on fire: How fact-checking influences citizens' reactions to negative advertising. Political Communication, 32(1), 127–151. https://doi.org/10.1080/10584609.2014.914613
Graves, L., & Cherubini, F. (2016). The rise of fact-checking sites in Europe. Reuters Institute for the Study of Journalism.
Graves, L. (2016). Deciding what's true: The rise of political fact-checking in American journalism. Columbia University Press.
Guo, Z., Schlichtkrull, M., & Vlachos, A. (2022). A survey on automated fact-checking. Transactions of the Association for Computational Linguistics, 10, 178–206. https://doi.org/10.1162/tacl_a_00454
Hassan, N., Arslan, F., Li, C., & Tremayne, M. (2017). Toward automated fact-checking: Detecting check-worthy factual claims by ClaimBuster. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 1803–1812). ACM. https://doi.org/10.1145/3097983.3098131
International Fact-Checking Network. (2016). The International Fact-Checking Network's code of principles. Poynter Institute.
Kovach, B., & Rosenstiel, T. (2014). The elements of journalism: What newspeople should know and the public should expect (3rd ed.). Three Rivers Press.
Lewandowsky, S., Ecker, U. K. H., Seifert, C. M., Schwarz, N., & Cook, J. (2012). Misinformation and its correction: Continued influence and successful debiasing. Psychological Science in the Public Interest, 13(3), 106–131. https://doi.org/10.1177/1529100612451018
Lim, C. (2018). Checking how fact-checkers check. Research & Politics, 5(3). https://doi.org/10.1177/2053168018786848
Marietta, M., Barker, D. C., & Bowser, T. (2015). Fact-checking polarized politics: Does the fact-check industry provide consistent guidance on disputed realities? The Forum, 13(4), 577–596. https://doi.org/10.1515/for-2015-0040
Nieminen, S., & Rapeli, L. (2019). Fighting misperceptions and doubting journalists' objectivity: A review of fact-checking literature. Political Studies Review, 17(3), 296–309. https://doi.org/10.1177/1478929918786852
Nyhan, B., Porter, E., Reifler, J., & Wood, T. J. (2020). Taking fact-checks literally but not seriously? The effects of journalistic fact-checking on factual beliefs and candidate favorability. Political Behavior, 42(3), 939–960. https://doi.org/10.1007/s11109-019-09528-x
Nyhan, B., & Reifler, J. (2015). The effect of fact-checking on elites: A field experiment on U.S. state legislators. American Journal of Political Science, 59(3), 628–640. https://doi.org/10.1111/ajps.12162
Nyhan, B., & Reifler, J. (2010). When corrections fail: The persistence of political misperceptions. Political Behavior, 32(2), 303–330. https://doi.org/10.1007/s11109-010-9112-2
Pennycook, G., Bear, A., Collins, E. T., & Rand, D. G. (2020). The implied truth effect: Attaching warnings to a subset of fake news headlines increases perceived accuracy of headlines without warnings. Management Science, 66(11), 4944–4957. https://doi.org/10.1287/mnsc.2019.3478
Porter, E., & Wood, T. J. (2021). The global effectiveness of fact-checking: Evidence from simultaneous experiments in Argentina, Nigeria, South Africa, and the United Kingdom. Proceedings of the National Academy of Sciences, 118(37), Article e2104235118. https://doi.org/10.1073/pnas.2104235118
Roozenbeek, J., & van der Linden, S. (2019). Fake news game confers psychological resistance against online misinformation. Palgrave Communications, 5, Article 65. https://doi.org/10.1057/s41599-019-0279-9
Shapiro, I., Brin, C., Bédard-Brûlé, I., & Mychajlowycz, K. (2013). Verification as a strategic ritual: How journalists retrospectively describe processes for ensuring accuracy. Journalism Practice, 7(6), 657–673. https://doi.org/10.1080/17512786.2013.765638
Silverman, C. (2007). Regret the error: How media mistakes pollute the press and imperil free speech. Union Square Press.
Thorne, J., Vlachos, A., Christodoulopoulos, C., & Mittal, A. (2018). FEVER: A large-scale dataset for fact extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 809–819). Association for Computational Linguistics. https://doi.org/10.18653/v1/N18-1074
Uscinski, J. E., & Butler, R. W. (2013). The epistemology of fact checking. Critical Review, 25(2), 162–180. https://doi.org/10.1080/08913811.2013.843872
Vosoughi, S., Roy, D., & Aral, S. (2018). The spread of true and false news online. Science, 359(6380), 1146–1151. https://doi.org/10.1126/science.aap9559
Walter, N., Cohen, J., Holbert, R. L., & Morag, Y. (2020). Fact-checking: A meta-analysis of what works and for whom. Political Communication, 37(3), 350–375. https://doi.org/10.1080/10584609.2019.1668894
Wood, T., & Porter, E. (2019). The elusive backfire effect: Mass attitudes' steadfast factual adherence. Political Behavior, 41(1), 135–163. https://doi.org/10.1007/s11109-018-9443-y