Skip to content

Research · Research design

Design-Based Implementation Research for Practice-Relevant Knowledge

Design-based implementation research brings researchers and practitioners together to design, test and refine solutions to persistent problems in real organisations. This article explains its principles, methods, evidence standards and lessons for applied research.

ModQu research article · 2,314 words · 22 references (APA 7th edition)

Download PDF

Much research is designed to answer whether an intervention works. Practitioners usually face a different question: how can we make something work here, with our people, budgets and constraints, and keep it working? Design-based implementation research (DBIR) was developed to answer that second question. It combines the iterative design methods of design-based research with the attention to organisations and systems found in implementation research. Researchers and practitioners work together on problems that practitioners care about, design solutions, test them in real settings and refine them over repeated cycles, while building both practical capacity and theory.

DBIR began in education, where researchers were frustrated that promising classroom innovations rarely spread or lasted. Its principles, however, apply to any field in which knowledge has to be put to work inside organisations: health services, public administration and business. This article sets out where DBIR came from, its core principles, how a project is organised, what counts as evidence, and its strengths and limitations.

Origins

Design-based research

In the early 1990s, Brown (1992) described "design experiments", in which researchers engineered innovative learning environments in real classrooms and studied them as they changed, rather than isolating single variables in a laboratory. Collins et al. (2004) set out the methodology of design research: it develops an intervention and a theory of how it works together, through cycles of design, enactment, analysis and redesign in naturalistic settings. The Design-Based Research Collective (2003) described the approach as an emerging paradigm whose central goals were to design learning environments and develop theories of learning at the same time.

Cobb et al. (2003) emphasised that design experiments are both "pragmatic" and "theoretical": they aim to improve practice and to test and generate theory about the processes involved. Barab and Squire (2004) argued that design-based research should be judged by its ability to produce usable designs and useful theory, not by the standards of controlled experiments alone.

Anderson and Shattuck (2012) reviewed a decade of design-based research and found that it had produced many promising interventions and insights, but that most studies stayed small, involved researchers in the design classrooms, and rarely examined how innovations worked when scaled up or when the researchers left. This was the gap DBIR set out to fill.

Implementation research

Implementation research, which grew mainly in health services, studies how evidence-based practices are adopted, delivered and sustained in routine settings. Fixsen et al. (2005) synthesised a large literature and concluded that well-tested programmes often failed in practice because implementation itself, including staff selection, training, coaching, data systems and organisational support, was neglected. Greenhalgh et al. (2004) reviewed research on the diffusion of innovations in service organisations and found that adoption depended on the innovation, the adopters, the organisation's readiness and the wider system, all interacting. Damschroder et al. (2009) integrated these factors into the Consolidated Framework for Implementation Research.

Bringing them together

Penuel et al. (2011) proposed DBIR as an approach at "the intersection of learning, implementation, and design". It takes from design-based research the commitment to iterative design and theory building, and from implementation research the focus on what it takes for innovations to work across many settings and over time.

Four principles

Penuel et al. (2011) described four core principles of DBIR, shown in Figure 1.

Four cards showing the principles of design-based implementation research: persistent problems of practice from several stakeholders' perspectives, iterative collaborative design, theory about learning and implementation, and capacity for sustaining change.
Figure 1. Four principles of design-based implementation research, after Penuel et al. (2011). Diagram by ModQu.

1. A focus on persistent problems of practice from multiple stakeholders' perspectives. Projects start from problems that practitioners recognise as important and enduring, not from a researcher's favourite intervention. Defining the problem involves people at different levels, such as front-line staff, managers and leaders, whose views of the problem often differ.

2. A commitment to iterative, collaborative design. Solutions are designed jointly by researchers and practitioners and refined through repeated cycles of trial and evaluation. Practitioners are co-designers, not merely sites or subjects.

3. A concern with developing theory and knowledge related to both learning and implementation through systematic inquiry. DBIR asks not only whether an intervention improves outcomes but also what conditions, supports and organisational arrangements make it work. Both kinds of knowledge are developed through systematic data collection and analysis.

4. A concern with developing capacity for sustaining change in systems. Projects aim to leave organisations better able to continue, adapt and spread improvements without researchers, for example by building data routines, training internal leaders and aligning policies.

How a DBIR project works

DBIR projects are organised in cycles that move between the specific setting and broader questions. Figure 2 shows a typical cycle.

Flow diagram of a DBIR cycle in four steps: define the problem jointly, co-design a solution, implement and study it in real settings, and refine and build capacity before the next cycle.
Figure 2. A design-based implementation research cycle. Each pass refines both the design and the conditions for implementing it. Diagram by ModQu.

Defining the problem

The partnership begins by agreeing on the problem. This often takes longer than expected, because stakeholders frame it differently. A school district may see low achievement in mathematics as a curriculum problem, while teachers see it as a problem of time and support. Data collected at this stage, such as interviews, observations and analysis of existing records, help the partners agree on a problem worth working on and on how improvement would be recognised.

Co-design

Researchers and practitioners design a solution together. Researchers contribute knowledge from previous research and methods for testing; practitioners contribute knowledge of the setting, constraints and what will be accepted. Co-design increases the chance that the design fits the setting and that practitioners feel ownership of it.

Implementation and study

The design is tried in real conditions, often first with a few teams or sites. Data are collected on outcomes and on implementation: how the design was used, by whom, with what adaptations and with what support. Mixed methods are typical, combining quantitative measures of outcomes with observation, interviews and documents.

Refinement and capacity building

Findings feed into revisions of the design and of the supports around it. As cycles proceed, the focus shifts from making the design work in a few places to making it work across the system, which raises new questions about training, leadership, policy and resources.

Improvement science

DBIR has close relatives. Bryk et al. (2015) described improvement science and networked improvement communities, in which organisations work together on a shared problem using disciplined cycles of testing. These use the Plan-Do-Study-Act cycle described by Langley et al. (2009): plan a small change and a prediction, try it, study the results against the prediction and act on what was learned. The approach shares DBIR's emphasis on learning through rapid, small tests in real settings, with stronger use of measurement and of variation across sites.

Partnerships

DBIR usually takes place within long-term research-practice partnerships. Coburn and Penuel (2016) reviewed evidence on such partnerships and found that they could improve the use of research in decision-making and, in some cases, outcomes, but that they faced challenges: differences in timelines, incentives and language between partners; turnover in practitioner organisations; and the effort needed to build trust. Penuel and Gallagher (2017) described practical steps for creating partnerships, including agreeing on shared aims, roles, data access and how findings will be published.

The partnership changes the researcher's role. Researchers are not detached observers but participants in change, which brings both access and responsibility. They must manage the tension between helping the partner succeed and reporting honestly when designs fail. The tradition of action research, in which Lewin (1946) proposed cycles of planning, action and fact-finding about the results of action, is an important ancestor of this stance.

Evidence and rigour

Because DBIR takes place in changing real settings, it cannot meet all the standards of controlled experiments. Its rigour comes from other sources.

McKenney and Reeves (2019) argued that educational design research should produce two kinds of output: a practical contribution, in the form of a tested design and guidance for its use, and a theoretical contribution, in the form of knowledge about how and why it works that is useful beyond the specific setting.

Scale and sustainability

DBIR treats scale as a problem to be studied, not something that happens after research finishes. Coburn (2003) argued that scale means more than spreading an innovation to many sites. It has four dimensions: depth, meaning real change in practice rather than surface features; sustainability, meaning that change lasts after initial support ends; spread, meaning that the ideas move to more sites and are embedded in policy and routines; and a shift in ownership, meaning that the organisation, rather than researchers, takes responsibility for the reform. DBIR projects use these dimensions to plan and evaluate their work.

Beyond education

DBIR's principles fit many applied settings. In health services, partnerships between researchers and clinics use similar cycles to adapt evidence-based practices. In information systems, design science research builds and evaluates artefacts, such as software, models and methods, to solve organisational problems, and requires that the artefact be rigorously evaluated and its contribution to knowledge made explicit (Hevner et al., 2004).

In business and market research, a DBIR approach might bring a research team together with customer service managers to address a persistent problem, such as complaints about delivery. The partners would define the problem with data from surveys and service records, co-design changes to processes and communications, test them in a few regions, measure both customer outcomes and how staff implemented the changes, and refine them before rolling out. The research produces a working solution and knowledge about what made it work.

DBIR also reflects the idea of use-inspired basic research described by Stokes (1997): it seeks fundamental understanding, for example of how organisations learn, while being driven by practical problems.

A worked example

Consider a hypothetical regional health authority whose clinics struggle to follow up patients after a positive screening test. Many patients never return for confirmatory testing. Researchers from a university and staff from the authority form a partnership.

In the first cycle, the partners examine clinic records and interview nurses, receptionists, managers and patients. Managers see the problem as patients failing to respond. Nurses see it as having no time to chase results. Patients report confusing letters and difficulty taking time off work. The partners agree on a shared problem statement and a measure: the share of patients completing follow-up within 60 days.

In the second cycle, they co-design a package: a plain-language letter, a text-message reminder, a named staff member responsible for follow-up lists and evening appointments one day a week. Three clinics try it. Follow-up improves in two clinics but not the third, where the named staff member left and no one took over the list.

In the third cycle, the partners revise the package to make follow-up a shared team task supported by a simple electronic list, and add training for clinic managers. The revised package is tried in eight clinics. Implementation data show which components were used and how they were adapted. The research produces a working package, evidence about the conditions under which it works and clinic staff able to run it without the researchers. A later randomised rollout across the region provides a stronger test of effectiveness.

Data and measurement

DBIR depends on data that practitioners can collect and use routinely, not only on data that researchers collect for a study. Partners often build practical measures: short, frequent indicators that track whether a change is happening and whether it is making a difference, such as weekly follow-up rates or brief surveys of staff. These measures support rapid learning within cycles and leave behind a data routine that the organisation can continue. More intensive research measures, such as validated scales, observations and interviews, are used at key points to test theory and to check that practical measures capture what matters.

Limitations

Practical guidance

  1. Start from a problem that practitioners own and care about.
  2. Involve stakeholders at several levels in defining the problem and designing solutions.
  3. State the conjectures each design embodies and how they will be tested.
  4. Collect implementation data as carefully as outcome data.
  5. Use short cycles to learn quickly, and stronger designs once a solution stabilises.
  6. Plan for capacity building and ownership from the start.
  7. Document every cycle, including failures.

Key takeaways

Design-based implementation research joins two traditions, design research and implementation research, to produce knowledge that is both rigorous and usable. It starts from practitioners' problems, designs solutions with them, studies how those solutions work in real systems and builds the capacity to sustain change. Its knowledge claims concern not only whether something works, but what it takes to make it work here and elsewhere.

References

Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education research? Educational Researcher, 41(1), 16–25. https://doi.org/10.3102/0013189X11428813

Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. Journal of the Learning Sciences, 13(1), 1–14. https://doi.org/10.1207/s15327809jls1301_1

Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. Journal of the Learning Sciences, 2(2), 141–178. https://doi.org/10.1207/s15327809jls0202_2

Bryk, A. S., Gomez, L. M., Grunow, A., & LeMahieu, P. G. (2015). Learning to improve: How America's schools can get better at getting better. Harvard Education Press.

Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9–13. https://doi.org/10.3102/0013189X032001009

Coburn, C. E., & Penuel, W. R. (2016). Research–practice partnerships in education: Outcomes, dynamics, and open questions. Educational Researcher, 45(1), 48–54. https://doi.org/10.3102/0013189X16631750

Coburn, C. E. (2003). Rethinking scale: Moving beyond numbers to deep and lasting change. Educational Researcher, 32(6), 3–12. https://doi.org/10.3102/0013189X032006003

Collins, A., Joseph, D., & Bielaczyc, K. (2004). Design research: Theoretical and methodological issues. Journal of the Learning Sciences, 13(1), 15–42. https://doi.org/10.1207/s15327809jls1301_2

Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M., & Stetler, C. (2012). Effectiveness-implementation hybrid designs: Combining elements of clinical effectiveness and implementation research to enhance public health impact. Medical Care, 50(3), 217–226. https://doi.org/10.1097/MLR.0b013e3182408812

Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, Article 50. https://doi.org/10.1186/1748-5908-4-50

Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5–8. https://doi.org/10.3102/0013189X032001005

Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. University of South Florida, Louis de la Parte Florida Mental Health Institute, National Implementation Research Network.

Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999). Evaluating the public health impact of health promotion interventions: The RE-AIM framework. American Journal of Public Health, 89(9), 1322–1327. https://doi.org/10.2105/AJPH.89.9.1322

Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., & Kyriakidou, O. (2004). Diffusion of innovations in service organizations: Systematic review and recommendations. The Milbank Quarterly, 82(4), 581–629. https://doi.org/10.1111/j.0887-378X.2004.00325.x

Hevner, A. R., March, S. T., Park, J., & Ram, S. (2004). Design science in information systems research. MIS Quarterly, 28(1), 75–105. https://doi.org/10.2307/25148625

Langley, G. J., Moen, R. D., Nolan, K. M., Nolan, T. W., Norman, C. L., & Provost, L. P. (2009). The improvement guide: A practical approach to enhancing organizational performance (2nd ed.). Jossey-Bass.

Lewin, K. (1946). Action research and minority problems. Journal of Social Issues, 2(4), 34–46. https://doi.org/10.1111/j.1540-4560.1946.tb02295.x

McKenney, S., & Reeves, T. C. (2019). Conducting educational design research (2nd ed.). Routledge. https://doi.org/10.4324/9781315105642

Penuel, W. R., & Gallagher, D. J. (2017). Creating research-practice partnerships in education. Harvard Education Press.

Penuel, W. R., Fishman, B. J., Cheng, B. H., & Sabelli, N. (2011). Organizing research and development at the intersection of learning, implementation, and design. Educational Researcher, 40(7), 331–337. https://doi.org/10.3102/0013189X11421826

Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38(2), 65–76. https://doi.org/10.1007/s10488-010-0319-7

Stokes, D. E. (1997). Pasteur's quadrant: Basic science and technological innovation. Brookings Institution Press.