Research · Research design
Design-Based Implementation Research for Practice-Relevant Knowledge
Design-based implementation research brings researchers and practitioners together to design, test and refine solutions to persistent problems in real organisations. This article explains its principles, methods, evidence standards and lessons for applied research.
Much research is designed to answer whether an intervention works. Practitioners usually face a different question: how can we make something work here, with our people, budgets and constraints, and keep it working? Design-based implementation research (DBIR) was developed to answer that second question. It combines the iterative design methods of design-based research with the attention to organisations and systems found in implementation research. Researchers and practitioners work together on problems that practitioners care about, design solutions, test them in real settings and refine them over repeated cycles, while building both practical capacity and theory.
DBIR began in education, where researchers were frustrated that promising classroom innovations rarely spread or lasted. Its principles, however, apply to any field in which knowledge has to be put to work inside organisations: health services, public administration and business. This article sets out where DBIR came from, its core principles, how a project is organised, what counts as evidence, and its strengths and limitations.
Origins
Design-based research
In the early 1990s, Brown (1992) described "design experiments", in which researchers engineered innovative learning environments in real classrooms and studied them as they changed, rather than isolating single variables in a laboratory. Collins et al. (2004) set out the methodology of design research: it develops an intervention and a theory of how it works together, through cycles of design, enactment, analysis and redesign in naturalistic settings. The Design-Based Research Collective (2003) described the approach as an emerging paradigm whose central goals were to design learning environments and develop theories of learning at the same time.
Cobb et al. (2003) emphasised that design experiments are both "pragmatic" and "theoretical": they aim to improve practice and to test and generate theory about the processes involved. Barab and Squire (2004) argued that design-based research should be judged by its ability to produce usable designs and useful theory, not by the standards of controlled experiments alone.
Anderson and Shattuck (2012) reviewed a decade of design-based research and found that it had produced many promising interventions and insights, but that most studies stayed small, involved researchers in the design classrooms, and rarely examined how innovations worked when scaled up or when the researchers left. This was the gap DBIR set out to fill.
Implementation research
Implementation research, which grew mainly in health services, studies how evidence-based practices are adopted, delivered and sustained in routine settings. Fixsen et al. (2005) synthesised a large literature and concluded that well-tested programmes often failed in practice because implementation itself, including staff selection, training, coaching, data systems and organisational support, was neglected. Greenhalgh et al. (2004) reviewed research on the diffusion of innovations in service organisations and found that adoption depended on the innovation, the adopters, the organisation's readiness and the wider system, all interacting. Damschroder et al. (2009) integrated these factors into the Consolidated Framework for Implementation Research.
Bringing them together
Penuel et al. (2011) proposed DBIR as an approach at "the intersection of learning, implementation, and design". It takes from design-based research the commitment to iterative design and theory building, and from implementation research the focus on what it takes for innovations to work across many settings and over time.
Four principles
Penuel et al. (2011) described four core principles of DBIR, shown in Figure 1.
1. A focus on persistent problems of practice from multiple stakeholders' perspectives. Projects start from problems that practitioners recognise as important and enduring, not from a researcher's favourite intervention. Defining the problem involves people at different levels, such as front-line staff, managers and leaders, whose views of the problem often differ.
2. A commitment to iterative, collaborative design. Solutions are designed jointly by researchers and practitioners and refined through repeated cycles of trial and evaluation. Practitioners are co-designers, not merely sites or subjects.
3. A concern with developing theory and knowledge related to both learning and implementation through systematic inquiry. DBIR asks not only whether an intervention improves outcomes but also what conditions, supports and organisational arrangements make it work. Both kinds of knowledge are developed through systematic data collection and analysis.
4. A concern with developing capacity for sustaining change in systems. Projects aim to leave organisations better able to continue, adapt and spread improvements without researchers, for example by building data routines, training internal leaders and aligning policies.
How a DBIR project works
DBIR projects are organised in cycles that move between the specific setting and broader questions. Figure 2 shows a typical cycle.
Defining the problem
The partnership begins by agreeing on the problem. This often takes longer than expected, because stakeholders frame it differently. A school district may see low achievement in mathematics as a curriculum problem, while teachers see it as a problem of time and support. Data collected at this stage, such as interviews, observations and analysis of existing records, help the partners agree on a problem worth working on and on how improvement would be recognised.
Co-design
Researchers and practitioners design a solution together. Researchers contribute knowledge from previous research and methods for testing; practitioners contribute knowledge of the setting, constraints and what will be accepted. Co-design increases the chance that the design fits the setting and that practitioners feel ownership of it.
Implementation and study
The design is tried in real conditions, often first with a few teams or sites. Data are collected on outcomes and on implementation: how the design was used, by whom, with what adaptations and with what support. Mixed methods are typical, combining quantitative measures of outcomes with observation, interviews and documents.
Refinement and capacity building
Findings feed into revisions of the design and of the supports around it. As cycles proceed, the focus shifts from making the design work in a few places to making it work across the system, which raises new questions about training, leadership, policy and resources.
Improvement science
DBIR has close relatives. Bryk et al. (2015) described improvement science and networked improvement communities, in which organisations work together on a shared problem using disciplined cycles of testing. These use the Plan-Do-Study-Act cycle described by Langley et al. (2009): plan a small change and a prediction, try it, study the results against the prediction and act on what was learned. The approach shares DBIR's emphasis on learning through rapid, small tests in real settings, with stronger use of measurement and of variation across sites.
Partnerships
DBIR usually takes place within long-term research-practice partnerships. Coburn and Penuel (2016) reviewed evidence on such partnerships and found that they could improve the use of research in decision-making and, in some cases, outcomes, but that they faced challenges: differences in timelines, incentives and language between partners; turnover in practitioner organisations; and the effort needed to build trust. Penuel and Gallagher (2017) described practical steps for creating partnerships, including agreeing on shared aims, roles, data access and how findings will be published.
The partnership changes the researcher's role. Researchers are not detached observers but participants in change, which brings both access and responsibility. They must manage the tension between helping the partner succeed and reporting honestly when designs fail. The tradition of action research, in which Lewin (1946) proposed cycles of planning, action and fact-finding about the results of action, is an important ancestor of this stance.
Evidence and rigour
Because DBIR takes place in changing real settings, it cannot meet all the standards of controlled experiments. Its rigour comes from other sources.
- Systematic documentation. Each cycle records the design, the setting, what was done and what was found, so that changes can be traced and others can learn from them.
- Multiple sources of evidence. Outcomes, implementation data and participants' accounts are combined.
- Theory testing. Each design embodies conjectures about how and why it should work; the data test and refine these conjectures.
- Implementation outcomes. Proctor et al. (2011) proposed distinct implementation outcomes, such as acceptability, adoption, appropriateness, feasibility, fidelity, cost, penetration and sustainability, which are measured alongside effectiveness.
- Rigorous tests when mature. Once a design has stabilised, it can be evaluated with stronger designs, including randomised trials. Curran et al. (2012) described hybrid designs that test effectiveness and implementation strategies together.
- Evaluation of reach and maintenance. The RE-AIM framework of Glasgow et al. (1999) assesses reach, effectiveness, adoption, implementation and maintenance, all central concerns of DBIR.
McKenney and Reeves (2019) argued that educational design research should produce two kinds of output: a practical contribution, in the form of a tested design and guidance for its use, and a theoretical contribution, in the form of knowledge about how and why it works that is useful beyond the specific setting.
Scale and sustainability
DBIR treats scale as a problem to be studied, not something that happens after research finishes. Coburn (2003) argued that scale means more than spreading an innovation to many sites. It has four dimensions: depth, meaning real change in practice rather than surface features; sustainability, meaning that change lasts after initial support ends; spread, meaning that the ideas move to more sites and are embedded in policy and routines; and a shift in ownership, meaning that the organisation, rather than researchers, takes responsibility for the reform. DBIR projects use these dimensions to plan and evaluate their work.
Beyond education
DBIR's principles fit many applied settings. In health services, partnerships between researchers and clinics use similar cycles to adapt evidence-based practices. In information systems, design science research builds and evaluates artefacts, such as software, models and methods, to solve organisational problems, and requires that the artefact be rigorously evaluated and its contribution to knowledge made explicit (Hevner et al., 2004).
In business and market research, a DBIR approach might bring a research team together with customer service managers to address a persistent problem, such as complaints about delivery. The partners would define the problem with data from surveys and service records, co-design changes to processes and communications, test them in a few regions, measure both customer outcomes and how staff implemented the changes, and refine them before rolling out. The research produces a working solution and knowledge about what made it work.
DBIR also reflects the idea of use-inspired basic research described by Stokes (1997): it seeks fundamental understanding, for example of how organisations learn, while being driven by practical problems.
A worked example
Consider a hypothetical regional health authority whose clinics struggle to follow up patients after a positive screening test. Many patients never return for confirmatory testing. Researchers from a university and staff from the authority form a partnership.
In the first cycle, the partners examine clinic records and interview nurses, receptionists, managers and patients. Managers see the problem as patients failing to respond. Nurses see it as having no time to chase results. Patients report confusing letters and difficulty taking time off work. The partners agree on a shared problem statement and a measure: the share of patients completing follow-up within 60 days.
In the second cycle, they co-design a package: a plain-language letter, a text-message reminder, a named staff member responsible for follow-up lists and evening appointments one day a week. Three clinics try it. Follow-up improves in two clinics but not the third, where the named staff member left and no one took over the list.
In the third cycle, the partners revise the package to make follow-up a shared team task supported by a simple electronic list, and add training for clinic managers. The revised package is tried in eight clinics. Implementation data show which components were used and how they were adapted. The research produces a working package, evidence about the conditions under which it works and clinic staff able to run it without the researchers. A later randomised rollout across the region provides a stronger test of effectiveness.
Data and measurement
DBIR depends on data that practitioners can collect and use routinely, not only on data that researchers collect for a study. Partners often build practical measures: short, frequent indicators that track whether a change is happening and whether it is making a difference, such as weekly follow-up rates or brief surveys of staff. These measures support rapid learning within cycles and leave behind a data routine that the organisation can continue. More intensive research measures, such as validated scales, observations and interviews, are used at key points to test theory and to check that practical measures capture what matters.
Limitations
- Time and cost. Partnerships and multiple cycles require years of work and stable funding.
- Generalisability. Designs refined in one system may depend on features of that system. Clear documentation and theory help, but transfer must be tested.
- Researcher involvement. Close involvement can bias interpretation. Independent evaluation at later stages and transparent reporting of failures guard against this.
- Publication. Iterative, context-rich studies are harder to publish in journals that expect a single, clean test.
- Power and voice. Partnerships can be dominated by leaders or researchers; deliberate effort is needed to include front-line staff and the people affected.
Practical guidance
- Start from a problem that practitioners own and care about.
- Involve stakeholders at several levels in defining the problem and designing solutions.
- State the conjectures each design embodies and how they will be tested.
- Collect implementation data as carefully as outcome data.
- Use short cycles to learn quickly, and stronger designs once a solution stabilises.
- Plan for capacity building and ownership from the start.
- Document every cycle, including failures.
Key takeaways
Design-based implementation research joins two traditions, design research and implementation research, to produce knowledge that is both rigorous and usable. It starts from practitioners' problems, designs solutions with them, studies how those solutions work in real systems and builds the capacity to sustain change. Its knowledge claims concern not only whether something works, but what it takes to make it work here and elsewhere.
References
Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education research? Educational Researcher, 41(1), 16–25. https://doi.org/10.3102/0013189X11428813
Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. Journal of the Learning Sciences, 13(1), 1–14. https://doi.org/10.1207/s15327809jls1301_1
Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. Journal of the Learning Sciences, 2(2), 141–178. https://doi.org/10.1207/s15327809jls0202_2
Bryk, A. S., Gomez, L. M., Grunow, A., & LeMahieu, P. G. (2015). Learning to improve: How America's schools can get better at getting better. Harvard Education Press.
Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9–13. https://doi.org/10.3102/0013189X032001009
Coburn, C. E., & Penuel, W. R. (2016). Research–practice partnerships in education: Outcomes, dynamics, and open questions. Educational Researcher, 45(1), 48–54. https://doi.org/10.3102/0013189X16631750
Coburn, C. E. (2003). Rethinking scale: Moving beyond numbers to deep and lasting change. Educational Researcher, 32(6), 3–12. https://doi.org/10.3102/0013189X032006003
Collins, A., Joseph, D., & Bielaczyc, K. (2004). Design research: Theoretical and methodological issues. Journal of the Learning Sciences, 13(1), 15–42. https://doi.org/10.1207/s15327809jls1301_2
Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M., & Stetler, C. (2012). Effectiveness-implementation hybrid designs: Combining elements of clinical effectiveness and implementation research to enhance public health impact. Medical Care, 50(3), 217–226. https://doi.org/10.1097/MLR.0b013e3182408812
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, Article 50. https://doi.org/10.1186/1748-5908-4-50
Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5–8. https://doi.org/10.3102/0013189X032001005
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. University of South Florida, Louis de la Parte Florida Mental Health Institute, National Implementation Research Network.
Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999). Evaluating the public health impact of health promotion interventions: The RE-AIM framework. American Journal of Public Health, 89(9), 1322–1327. https://doi.org/10.2105/AJPH.89.9.1322
Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., & Kyriakidou, O. (2004). Diffusion of innovations in service organizations: Systematic review and recommendations. The Milbank Quarterly, 82(4), 581–629. https://doi.org/10.1111/j.0887-378X.2004.00325.x
Hevner, A. R., March, S. T., Park, J., & Ram, S. (2004). Design science in information systems research. MIS Quarterly, 28(1), 75–105. https://doi.org/10.2307/25148625
Langley, G. J., Moen, R. D., Nolan, K. M., Nolan, T. W., Norman, C. L., & Provost, L. P. (2009). The improvement guide: A practical approach to enhancing organizational performance (2nd ed.). Jossey-Bass.
Lewin, K. (1946). Action research and minority problems. Journal of Social Issues, 2(4), 34–46. https://doi.org/10.1111/j.1540-4560.1946.tb02295.x
McKenney, S., & Reeves, T. C. (2019). Conducting educational design research (2nd ed.). Routledge. https://doi.org/10.4324/9781315105642
Penuel, W. R., & Gallagher, D. J. (2017). Creating research-practice partnerships in education. Harvard Education Press.
Penuel, W. R., Fishman, B. J., Cheng, B. H., & Sabelli, N. (2011). Organizing research and development at the intersection of learning, implementation, and design. Educational Researcher, 40(7), 331–337. https://doi.org/10.3102/0013189X11421826
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38(2), 65–76. https://doi.org/10.1007/s10488-010-0319-7
Stokes, D. E. (1997). Pasteur's quadrant: Basic science and technological innovation. Brookings Institution Press.