Skip to content

Research · Communicating research

From Evidence to Decision: Research Pathways for Policy Knowledge

How research evidence reaches policy decisions, the different ways policy makers use research, structured evidence-to-decision frameworks, what helps and hinders the use of evidence, and practical steps for researchers who want their work to inform policy.

ModQu research article · 2,185 words · 18 references (APA 7th edition)

Download PDF

Governments spend large sums on research and on programmes that research is meant to inform. Yet the path from a research finding to a policy decision is rarely direct. Decisions are made under time pressure, with incomplete information, and with values, budgets and public opinion in play alongside evidence. This article explains how research enters policy, the different ways it is used, the structured frameworks that have been developed to link evidence to decisions, what the research on research use shows, and what researchers can do to make their work useful to those who decide.

Evidence-based and evidence-informed policy

The phrase evidence-based policy became common in the late 1990s, borrowing from evidence-based medicine. Davies et al. (2000) brought together accounts of how evidence was being used in British public services such as health care, education, criminal justice and social care, and showed that the types of evidence available, and the willingness to use them, varied widely between sectors.

Many researchers now prefer the term evidence-informed policy. Cairney and Oliver (2017) argue that policy making differs from clinical medicine in important ways. A clinical guideline can often rest on a hierarchy of evidence with randomised trials at the top. A policy decision usually involves competing goals, many affected groups and outcomes that cannot be tested in trials, and it is made within a political system in which elected officials are entitled to weigh evidence against values. Parkhurst (2017) makes a similar point: evidence can be used well or badly in policy, but deciding what to do always involves judgements about what matters, which evidence alone cannot settle. In this view, the aim is for decisions to be informed by the best available evidence, not determined by it.

How research is used

Weiss (1979) identified several distinct meanings of "research utilisation", which remain the most widely cited classification. They include:

Weiss argued that the enlightenment model described much of the real influence of social science. Research rarely determines a specific decision, but over time it can change which problems are considered important and which solutions are considered possible. Nutley et al. (2007) build on this distinction between instrumental use, where research directly shapes a decision, and conceptual use, where it changes understanding. They note that conceptual use is harder to observe but often more lasting.

When evidence can enter

Policy is often described as a cycle of stages: identifying a problem, developing options, deciding, implementing and evaluating. Real policy making is less orderly, but the stages are a useful guide to the different questions research can answer at each point.

Five-stage policy cycle with the research question asked at each stage: problem identification asks how big the problem is and who is affected; option development asks what works and at what cost; decision asks about the balance of benefits, harms, costs, values and feasibility; implementation asks whether it is delivered as intended; evaluation asks whether it achieved its aims. An arrow returns from evaluation to problem identification.
Figure 1. Different research questions arise at different stages of policy making. The stages overlap in practice. Based on Nutley et al. (2007) and Cairney (2016); diagram by ModQu.

Problem identification. Descriptive research, such as surveys and administrative data, shows how large a problem is, whom it affects and whether it is growing. Kingdon (2011) describes how issues reach the policy agenda when three streams come together: a recognised problem, an available policy solution, and favourable political conditions. Research contributes most clearly to the first two streams, by defining problems and by developing and testing solutions before the political opportunity arrives.

Developing options. Evaluations and systematic reviews show which interventions have worked elsewhere, for whom and at what cost. Systematic reviews are particularly valuable because single studies can mislead, and because they bring together the evidence in a form that busy officials can use (Lavis et al., 2005).

Deciding. At the point of decision, evidence on effects must be combined with information on costs, feasibility, public acceptability and fairness. Structured frameworks, described below, help make this process explicit.

Implementation and evaluation. Research on implementation shows whether a programme is delivered as intended, and evaluation shows whether it achieved its aims. Haynes et al. (2012), writing for the UK Cabinet Office, argued that government should test new policies with randomised trials where possible, learn from the results and adapt policies accordingly.

Evidence-to-decision frameworks

One way to link evidence and decisions transparently is to use a structured framework that sets out each consideration in turn. The best-known is the GRADE Evidence to Decision (EtD) framework, developed for health care recommendations and coverage decisions (Alonso-Coello et al., 2016). It asks decision makers to make an explicit judgement on a set of criteria, including:

For each criterion, the framework records the research evidence, any additional considerations and the panel's judgement, and then the resulting recommendation. The approach does not remove judgement. It makes the judgement visible, so that others can see which evidence was considered, where it was uncertain, and how values and practical constraints affected the final decision. The same logic can be applied outside health care, for example in an options paper for a minister that sets out, for each option, the evidence on effects, costs, risks and feasibility.

What helps and what hinders

A substantial body of research has asked why research is or is not used by policy makers. Innvær et al. (2002) reviewed interview studies with health policy makers and found that the factors most often reported as helping research use were personal contact between researchers and policy makers, research that was timely and relevant, and summaries that included clear recommendations. Oliver et al. (2014) updated this review with 145 newer studies and reached similar conclusions. The most commonly reported barriers were poor access to good-quality, relevant research and a lack of timely research output. The most commonly reported facilitators were collaboration between researchers and policy makers, and improved relationships and skills.

These findings point to differences between the two communities. Researchers often work to the timetable of grants and journals, which can take years; policy makers may need an answer in weeks. Researchers tend to emphasise uncertainty and the need for more research; policy makers need to act on the evidence available. Lomas (2007) described the role of knowledge brokers, people and organisations who work between the two communities, helping to frame questions that research can answer and translating findings into forms decision makers can use.

Lavis et al. (2003) proposed that research organisations plan their knowledge transfer by answering five questions: what should be transferred, to whom, by whom, how, and with what effect. They argued that the message should usually be an actionable conclusion from a body of research rather than the results of a single study, that the audience should be identified specifically, and that the messenger should be someone the audience trusts.

A cautionary example: effects at scale

Evidence that looks strong in published studies does not always hold when policies are carried out at scale. DellaVigna and Linos (2022) compared two sets of "nudge" interventions, such as reminder letters and simplified forms designed to increase take-up of services. In trials published in academic journals, nudges increased take-up by an average of 8.7 percentage points. In 126 trials run by two large government nudge units in the United States, covering more than 23 million people, the average effect was 1.4 percentage points: still statistically significant and useful, but much smaller.

Bar chart comparing average nudge effects on take-up: 8.7 percentage points in academic journal trials, a 33.5% increase over the control group, versus 1.4 percentage points in 126 nudge unit trials covering over 23 million people, an 8.1% increase.
Figure 2. Average effect of nudge interventions on take-up in published academic trials and in trials run by two US government nudge units. Source: DellaVigna and Linos (2022). Chart by ModQu.

The authors found that publication bias, made worse by small samples in many academic trials, could account for the gap; differences in the type of intervention also played a part. For policy makers, the lesson is that effect sizes from the published literature may overstate what a policy will achieve in practice. Systematic reviews that account for publication bias, pilot testing and evaluation at scale help to set realistic expectations.

The role of the researcher

Researchers who want to inform policy face choices about their role. Pielke (2007) described four roles a scientist can take in relation to policy. The pure scientist provides findings without engaging with decision makers. The science arbiter answers specific factual questions put by decision makers. The issue advocate argues for a particular course of action. The honest broker of policy alternatives sets out the range of available options and their likely consequences, leaving the choice to decision makers. Pielke argued that each role can be legitimate, but that researchers should be clear about which one they are taking, because presenting advocacy as neutral science damages trust in research.

Oliver and Cairney (2019) reviewed published advice to academics on how to influence policy. Common recommendations included producing high-quality research, making it relevant and readable, understanding how the policy process works, engaging with policy makers regularly, building relationships and being clear whether one is acting as an advocate or an honest broker. They also found that much of the advice was based on personal experience rather than evidence, and that it often treated policy making as more orderly and receptive to research than it is.

Practical steps for researchers

The evidence on research use suggests several practical steps.

Ask policy-relevant questions from the start. Involving policy makers and practitioners in shaping research questions increases the chance that the results will answer questions they actually face. Oliver et al. (2014) found collaboration to be the most frequently reported facilitator of use.

Synthesise, not only report. Decision makers usually need to know what the body of evidence shows, not what one new study found. Systematic reviews and evidence summaries are more useful to them than single studies (Lavis et al., 2005).

Write for the reader. A policy brief should state the main conclusion first, explain what it means for the decision at hand, give the strength of the evidence plainly, and keep technical detail for an appendix. Summaries with clear implications were among the facilitators most often reported by policy makers (Innvær et al., 2002).

Be honest about uncertainty. Policy makers need to know how confident they can be. Stating the certainty of the evidence, as the GRADE EtD framework requires, is more useful than presenting either false precision or a general call for more research.

Time the work to the decision. Evidence that arrives after a decision has been made has little effect on it. Researchers can follow policy timetables, such as budget cycles, consultations and legislative reviews, and offer interim findings where appropriate.

Build relationships. Personal contact was the facilitator most often reported in early reviews (Innvær et al., 2002). Relationships built over time, through advisory roles, secondments or regular meetings, allow researchers to understand the questions policy makers face and allow policy makers to know whom to ask.

Plan for evaluation. When a new policy is introduced, building an evaluation into its design, for example by phasing it in so that early and later areas can be compared, generates evidence for the next decision (Haynes et al., 2012).

The limits of evidence

Evidence cannot settle questions that are about values. Whether to prioritise economic growth or environmental protection, or how to balance individual liberty against public health, are political choices. Research can show the likely consequences of each option, but deciding among them is the responsibility of elected governments and, through them, the public (Parkhurst, 2017). Head (2016) notes that the use of evidence also depends on the capacity of government agencies, including their analytical skills, data systems and relationships with researchers.

Evidence can also be misused, selected to support a decision already made, or ignored. Weiss's (1979) political and tactical uses describe these patterns. Transparent frameworks, published evidence reviews and independent evaluation make misuse easier to detect, which is one reason they matter.

Summary

Research reaches policy through several routes: directly, by informing specific decisions, and indirectly, by changing how problems are understood. Different research questions arise at each stage of the policy process. Evidence-to-decision frameworks make the link between evidence, values and practical constraints explicit. Reviews of research use consistently find that timely, relevant research, collaboration and relationships between researchers and policy makers, and clear summaries help evidence to be used. Researchers who want to inform policy should design their work with decision makers in mind, synthesise evidence, state uncertainty honestly and be clear about the role they are taking.

References

Alonso-Coello, P., Schünemann, H. J., Moberg, J., Brignardello-Petersen, R., Akl, E. A., Davoli, M., Treweek, S., Mustafa, R. A., Rada, G., Rosenbaum, S., Morelli, A., Guyatt, G. H., Oxman, A. D., & GRADE Working Group. (2016). GRADE Evidence to Decision (EtD) frameworks: A systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ, 353, Article i2016. https://doi.org/10.1136/bmj.i2016

Cairney, P. (2016). The politics of evidence-based policy making. Palgrave Macmillan.

Cairney, P., & Oliver, K. (2017). Evidence-based policymaking is not like evidence-based medicine, so how far should you go to bridge the divide between evidence and policy? Health Research Policy and Systems, 15, Article 35. https://doi.org/10.1186/s12961-017-0192-x

Davies, H. T. O., Nutley, S. M., & Smith, P. C. (Eds.). (2000). What works? Evidence-based policy and practice in public services. Policy Press.

DellaVigna, S., & Linos, E. (2022). RCTs to scale: Comprehensive evidence from two nudge units. Econometrica, 90(1), 81–116. https://doi.org/10.3982/ECTA18709

Haynes, L., Service, O., Goldacre, B., & Torgerson, D. (2012). Test, learn, adapt: Developing public policy with randomised controlled trials. Cabinet Office Behavioural Insights Team.

Head, B. W. (2016). Toward more "evidence-informed" policy making? Public Administration Review, 76(3), 472–484. https://doi.org/10.1111/puar.12475

Innvær, S., Vist, G., Trommald, M., & Oxman, A. (2002). Health policy-makers' perceptions of their use of evidence: A systematic review. Journal of Health Services Research & Policy, 7(4), 239–244. https://doi.org/10.1258/135581902320432778

Kingdon, J. W. (2011). Agendas, alternatives, and public policies (Updated 2nd ed.). Longman.

Lavis, J., Davies, H., Oxman, A., Denis, J.-L., Golden-Biddle, K., & Ferlie, E. (2005). Towards systematic reviews that inform health care management and policy-making. Journal of Health Services Research & Policy, 10(Suppl. 1), 35–48. https://doi.org/10.1258/1355819054308549

Lavis, J. N., Robertson, D., Woodside, J. M., McLeod, C. B., & Abelson, J. (2003). How can research organizations more effectively transfer research knowledge to decision makers? The Milbank Quarterly, 81(2), 221–248. https://doi.org/10.1111/1468-0009.t01-1-00052

Lomas, J. (2007). The in-between world of knowledge brokering. BMJ, 334(7585), 129–132. https://doi.org/10.1136/bmj.39038.593380.AE

Nutley, S. M., Walter, I., & Davies, H. T. O. (2007). Using evidence: How research can inform public services. Policy Press.

Oliver, K., & Cairney, P. (2019). The dos and don'ts of influencing policy: A systematic review of advice to academics. Palgrave Communications, 5, Article 21. https://doi.org/10.1057/s41599-019-0232-y

Oliver, K., Innvar, S., Lorenc, T., Woodman, J., & Thomas, J. (2014). A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Services Research, 14, Article 2. https://doi.org/10.1186/1472-6963-14-2

Parkhurst, J. (2017). The politics of evidence: From evidence-based policy to the good governance of evidence. Routledge.

Pielke, R. A., Jr. (2007). The honest broker: Making sense of science in policy and politics. Cambridge University Press.

Weiss, C. H. (1979). The many meanings of research utilization. Public Administration Review, 39(5), 426–431. https://doi.org/10.2307/3109916