Skip to content

The Generation of Knowledge · Knowledge production

From Inquiry and Investigation to Defensible Knowledge

How inquiry and investigation turn questions into conclusions that can withstand challenge, using the structure of argument, the weighing of evidence, alternative explanations, pre-specified methods and full records.

ModQu research article · 1,050 words · 11 references (APA 7th edition)

Download PDF

Inquiry begins with a question; investigation follows evidence; and the aim of both is a conclusion that can be relied upon. In research, law, auditing and intelligence work, that conclusion must also be defensible: able to withstand scrutiny from someone who doubts it, by showing what evidence supports it, how the evidence was gathered and why other conclusions were rejected. This article sets out what makes knowledge defensible and how inquiry and investigation can be organised to produce it.

Ways of settling belief

The philosopher Charles Sanders Peirce described four ways in which people settle their beliefs (Peirce, 1877). The method of tenacity clings to what one already believes. The method of authority accepts what an institution declares. The a priori method adopts what seems reasonable or agreeable. The method of science tests beliefs against something independent of anyone's opinion: the facts of experience, investigated in a way that others can repeat. Only the last, Peirce argued, allows beliefs to be corrected when they are wrong.

Dewey (1938) described inquiry as the controlled transformation of an uncertain situation into a settled one. Inquiry begins when something is doubtful or problematic, proceeds by forming ideas about what might resolve it and testing them against observation, and ends in a warranted conclusion, which Dewey preferred to call a warranted assertion to emphasise that it remains open to further inquiry.

The structure of a defensible claim

Toulmin (1958) analysed the structure of practical arguments. A claim is supported by data, the evidence offered. A warrant explains why the data support the claim, and backing supports the warrant. A qualifier states how strongly the claim is made, such as "probably" or "in most cases", and a rebuttal names the conditions under which the claim would not hold. A defensible conclusion makes each element explicit, so that a critic can see exactly where to challenge it.

Diagram of Toulmin's model of argument applied to a survey finding: data lead through a qualifier to a claim; a warrant, supported by backing, explains why the data support the claim; and a rebuttal states when the claim would fail.
Figure 1. The structure of a defensible claim, with a survey example. Based on Toulmin (1958); diagram by ModQu.

Figure 1 applies the model to a survey finding. The data are the survey result; the warrant is the principle that a random sample represents the population within a known margin of error; the backing is sampling theory together with checks on who responded; the qualifier states the precision; and the rebuttal names the main condition under which the claim would fail, namely that the people who answered differ in relevant ways from those who did not. Written out in this way, the argument shows a sceptical reader exactly what would need to be true for the conclusion to be wrong, and what evidence would settle the matter.

Inquiry and investigation

Inquiry and investigation are related but not identical. Inquiry is open: it begins with a question and may lead in directions no one expected. Investigation is directed: it begins with a specific matter, such as a complaint, an anomaly in the data or a suspected cause, and follows evidence to establish what happened. Both need the same discipline. Evidence must be collected in ways that preserve its integrity, recorded so that its origin can be traced, and examined by people prepared to accept whatever it shows. Auditors and forensic investigators speak of a chain of custody, the documented record of who handled each item of evidence and when, so that no one can later claim it was altered. Research data need an equivalent: a record linking every published figure to the raw data and the steps that produced it.

Weighing evidence

Investigations rarely find a single decisive piece of evidence. Usually they assemble many items, each partial and some conflicting. Schum (1994) analysed how such evidence can be weighed, distinguishing its relevance (does it bear on the question?), its credibility (can its source be trusted?) and its inferential force (how much does it change the probability of the conclusion?). Anderson et al. (2005) set out methods used in legal reasoning for charting how items of evidence connect to the propositions that must be proved, so that gaps and weak links become visible.

Haack (1993) compared the justification of a belief to a crossword puzzle: an entry is well supported when it fits its clue (the direct evidence) and also intersects consistently with other completed entries (the rest of what we know). Defensible conclusions are supported in both ways.

Considering alternatives

A conclusion is only as strong as the case against its rivals. Popper (1959) argued that knowledge advances by subjecting claims to tests that could refute them. Heuer (1999) turned this into a practical technique for intelligence analysts, the analysis of competing hypotheses: list every plausible explanation, assess each item of evidence against every explanation, and favour the explanation with the least evidence against it, rather than the one with the most evidence apparently in its favour. Much evidence is consistent with several explanations and therefore distinguishes none of them.

Protecting the process

Defensibility depends on how conclusions were reached, not only on the conclusions themselves. Simmons et al. (2011) showed that common flexible practices, such as trying several analyses, adding observations until a result appears or reporting only some measures, can make it easy to find apparently significant results that are false. Ioannidis (2005) argued that, under realistic conditions of small studies, flexible designs and many tested relationships, a large share of published findings are likely to be false. Nosek et al. (2018) recommended preregistration: recording the questions, design and planned analysis before seeing the data, so that confirmatory tests can be distinguished from exploration.

Practical rules

Conclusion

Inquiry and investigation produce defensible knowledge when they are open about their evidence, their reasoning and their limits. A defensible conclusion does not claim certainty. It shows its grounds, considers its rivals, states its strength and invites challenge. That is what allows others, including sceptical clients, regulators and competitors, to rely on it.

References

Anderson, T., Schum, D., & Twining, W. (2005). Analysis of evidence (2nd ed.). Cambridge University Press.

Dewey, J. (1938). Logic: The theory of inquiry. Henry Holt.

Haack, S. (1993). Evidence and inquiry: Towards reconstruction in epistemology. Blackwell.

Heuer, R. J., Jr. (1999). Psychology of intelligence analysis. Center for the Study of Intelligence, Central Intelligence Agency.

Ioannidis, J. P. A. (2005). Why most published research findings are false. PLOS Medicine, 2(8), Article e124. https://doi.org/10.1371/journal.pmed.0020124

Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600–2606. https://doi.org/10.1073/pnas.1708274114

Peirce, C. S. (1877). The fixation of belief. Popular Science Monthly, 12, 1–15.

Popper, K. R. (1959). The logic of scientific discovery. Hutchinson.

Schum, D. A. (1994). The evidential foundations of probabilistic reasoning. Wiley.

Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632

Toulmin, S. E. (1958). The uses of argument. Cambridge University Press.