Skip to content

Research · Quantitative and mixed methods

Desk Research: Finding and Evaluating Published Sources

How to carry out desk research, covering the main types of published sources, how to plan and run searches, how to find grey literature, how to evaluate credibility and bias, and how to record and synthesise what is found.

ModQu research article · 1,658 words · 17 references (APA 7th edition)

Download PDF

Before collecting new data, a researcher should find out what is already known. Desk research is the systematic search for, and evaluation of, information that has already been published: academic articles, books, reports, statistics, news, company documents and websites. It is sometimes called secondary research, though it differs from the reanalysis of datasets. Stewart and Kamins (1993) described it as the use of existing information to answer questions, and noted that it is often the fastest and cheapest way to learn about a topic, to frame a primary study or to answer a question outright. This article explains how to find published sources and, just as important, how to judge whether they can be trusted.

Why desk research comes first

Desk research has several uses:

Hart (2018) argued that a review of existing work is the foundation of any research project, because it shows the researcher what is known, what is disputed and where the gaps lie.

Types of published sources

Published sources differ in how they are produced and checked. The table below summarises the main types.

Table of six types of published sources: peer-reviewed research, books and reference works, official publications, grey literature, news and trade media, and company documents, showing how each is checked and its main caution.
Figure 1. Types of published sources used in desk research. Diagram by ModQu, drawing on Stewart and Kamins (1993) and Paez (2017).

Peer-reviewed research in academic journals has been checked by other experts before publication. It is usually the most reliable source on research questions, though peer review does not guarantee correctness.

Books and reference works give broader overviews and summaries of established knowledge.

Official publications from governments, statistical agencies and international organisations provide statistics and policy information.

Grey literature is material produced outside commercial and academic publishing: reports by think tanks, charities, consultancies and industry bodies, conference papers, theses, working papers and government reports. Paez (2017) noted that grey literature often contains recent and specialised information not yet available in journals, and that excluding it can bias a review toward published, positive findings. Adams et al. (2017) described how grey literature is especially important in management and organisational research, where much practical knowledge is held in practitioner reports.

News and trade media report developments quickly but vary widely in accuracy.

Company and organisational documents, such as annual reports and press releases, give direct information about organisations but are written to present them favourably.

Planning a search

Good desk research starts with a plan. Booth et al. (2016) recommended defining the question clearly, identifying key concepts and their synonyms, deciding which sources to search and setting criteria for what will be included. A plan prevents both missing important sources and drowning in irrelevant ones.

A search string combines terms using Boolean operators: AND narrows a search to records containing all terms, OR widens it to records containing any of a set of synonyms, and NOT excludes terms. Truncation symbols find variant word endings. A search for customer loyalty in retail banking might combine terms for loyalty (loyalty OR retention OR churn) with terms for the sector (bank* OR "financial services").

Where to search

Different search tools cover different material. Academic databases index peer-reviewed journals in particular fields. Gusenbauer and Haddaway (2020) compared 28 academic search systems and found that they differed greatly in coverage and in the search functions they offered, and that only some were suitable as the main source for systematic searching. Martín-Martín et al. (2018) compared Google Scholar, Web of Science and Scopus and found that Google Scholar found the most citations in nearly all subject areas, but included more material of uncertain quality.

Haddaway et al. (2015) found that Google Scholar was useful for finding grey literature, but that its results could not be fully reproduced and only the first several hundred results could be viewed. They recommended using it alongside, not instead of, specialist databases.

Searching databases is only part of the process. Greenhalgh and Peacock (2005) audited the sources of studies included in a systematic review of complex evidence and found that only about a quarter came from database searches. Most were found by following references in other papers, by tracking which later papers cited key studies, and through personal knowledge and contacts. These methods, often called snowballing or citation chasing, are essential for thorough desk research.

Evaluating sources

Finding sources is only half the task. Each source must be evaluated before it is used. Several checklists help. Blakeslee (2004) proposed the CRAAP test, which asks about currency, relevance, authority, accuracy and purpose. Tyndall (2010) developed the AACODS checklist for grey literature, which asks about authority, accuracy, coverage, objectivity, date and significance. The questions they share are:

Checklists have limits. Wineburg and McGrew (2019) compared how professional fact-checkers, historians and students evaluated websites. Historians and students tended to read a site closely and judge it by its appearance and content. Fact-checkers instead left the site almost immediately and searched for what other sources said about it, a practice they called lateral reading. The fact-checkers reached accurate judgements faster. Metzger (2007) similarly found that people often judge online credibility by surface features such as design, rather than by the checks that experts recommend.

Bias in published sources

Several kinds of bias can distort what is published.

Publication bias. Studies with positive or striking results are more likely to be published than those with null results. Rothstein et al. (2005) showed how this distorts the evidence available to reviewers, making effects seem larger and more consistent than they are.

Sponsorship bias. Research funded by organisations with an interest in the outcome tends to favour that interest. Lundh et al. (2017), in a Cochrane review, found that studies of drugs and medical devices sponsored by manufacturers more often reported favourable results and conclusions than studies with other sources of funding.

Weak findings. Ioannidis (2005) argued that many published research findings are likely to be false, particularly when studies are small, effects are small, many relationships are tested and there is flexibility in design and analysis. A single study, however prominent, should be treated as provisional until it is supported by others.

Predatory publishing. Some journals charge authors to publish but provide little or no genuine peer review. Shen and Björk (2015) estimated that around 420,000 articles were published in such journals in 2014, spread across about 8,000 active journals, so researchers are likely to meet them in searches. Checking that a journal is indexed in recognised databases and that its editorial practices are clear helps to avoid them.

Knowing when to stop

Desk research can expand without limit, so it helps to decide in advance when the search is complete enough. Booth et al. (2016) suggested several signals: new searches return mostly sources already found, citation chasing leads back to the same core studies, and further sources add detail but do not change the answer. For a quick business question, a few authoritative sources may be enough; for a review that will guide major decisions or a primary study, a more exhaustive search is warranted. Whatever the scope, the researcher should state it, so that readers know how thoroughly the evidence was searched and how much weight the conclusions can bear.

A further check is to look for evidence against the emerging answer. Searching specifically for studies or reports that reach different conclusions, and reading them carefully, protects against building a case from only the sources that happen to agree.

Recording and synthesising

Good desk research keeps a record of what was searched, when and with what terms, so that the search can be repeated or updated. Reference management software helps store and organise sources. For each source, notes should record the key findings, the methods used and any concerns about quality.

Synthesis means bringing findings together rather than summarising sources one by one. Hart (2018) recommended organising material by theme or question, comparing what different sources say, and noting where they agree, where they disagree and why. A synthesis should weigh sources by their quality, not simply count them.

An applied example

A hypothetical example: a company considering a new subscription product wants to estimate how many households in its market already use similar services. Desk research finds a government household survey with relevant questions, two industry reports with differing estimates, and several news articles quoting a figure without a source. The researcher traces the news figure to one of the industry reports, finds that the report's estimate came from a small online panel run by a firm that sells the services, and gives it less weight. The government survey, though two years old, uses a large random sample and documented methods. The final estimate is based on the survey, adjusted for growth using the trend in the industry reports, with the uncertainty stated. The figures are illustrative, but the process shows how source evaluation changes the answer.

Conclusion

Desk research is often the first and sometimes the only research a question needs. Its value depends on searching widely and systematically, including grey literature and citation chasing as well as databases, and on evaluating every source for authority, purpose, method and bias. Recording the process and synthesising findings by theme turn a pile of documents into reliable knowledge.

References

Adams, R. J., Smart, P., & Huff, A. S. (2017). Shades of grey: Guidelines for working with the grey literature in systematic reviews for management and organizational studies. International Journal of Management Reviews, 19(4), 432–454. https://doi.org/10.1111/ijmr.12102

Blakeslee, S. (2004). The CRAAP test. LOEX Quarterly, 31(3), Article 4.

Booth, A., Sutton, A., & Papaioannou, D. (2016). Systematic approaches to a successful literature review (2nd ed.). SAGE.

Greenhalgh, T., & Peacock, R. (2005). Effectiveness and efficiency of search methods in systematic reviews of complex evidence: Audit of primary sources. BMJ, 331(7524), 1064–1065. https://doi.org/10.1136/bmj.38636.593461.68

Gusenbauer, M., & Haddaway, N. R. (2020). Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Research Synthesis Methods, 11(2), 181–217. https://doi.org/10.1002/jrsm.1378

Haddaway, N. R., Collins, A. M., Coughlin, D., & Kirk, S. (2015). The role of Google Scholar in evidence reviews and its applicability to grey literature searching. PLOS ONE, 10(9), Article e0138237. https://doi.org/10.1371/journal.pone.0138237

Hart, C. (2018). Doing a literature review: Releasing the research imagination (2nd ed.). SAGE.

Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2(8), Article e124. https://doi.org/10.1371/journal.pmed.0020124

Lundh, A., Lexchin, J., Mintzes, B., Schroll, J. B., & Bero, L. (2017). Industry sponsorship and research outcome. Cochrane Database of Systematic Reviews, (2), Article MR000033. https://doi.org/10.1002/14651858.MR000033.pub3

Martín-Martín, A., Orduna-Malea, E., Thelwall, M., & Delgado López-Cózar, E. (2018). Google Scholar, Web of Science, and Scopus: A systematic comparison of citations in 252 subject categories. Journal of Informetrics, 12(4), 1160–1177. https://doi.org/10.1016/j.joi.2018.09.002

Metzger, M. J. (2007). Making sense of credibility on the Web: Models for evaluating online information and recommendations for future research. Journal of the American Society for Information Science and Technology, 58(13), 2078–2091. https://doi.org/10.1002/asi.20672

Paez, A. (2017). Gray literature: An important resource in systematic reviews. Journal of Evidence-Based Medicine, 10(3), 233–240. https://doi.org/10.1111/jebm.12266

Rothstein, H. R., Sutton, A. J., & Borenstein, M. (Eds.). (2005). Publication bias in meta-analysis: Prevention, assessment and adjustments. Wiley. https://doi.org/10.1002/0470870168

Shen, C., & Björk, B.-C. (2015). 'Predatory' open access: A longitudinal study of article volumes and market characteristics. BMC Medicine, 13, Article 230. https://doi.org/10.1186/s12916-015-0469-2

Stewart, D. W., & Kamins, M. A. (1993). Secondary research: Information sources and methods (2nd ed.). SAGE.

Tyndall, J. (2010). AACODS checklist. Flinders University.

Wineburg, S., & McGrew, S. (2019). Lateral reading and the nature of expertise: Reading less and learning more when evaluating digital information. Teachers College Record, 121(11), 1–40. https://doi.org/10.1177/016146811912101102