Research · Market and business research
Rapid Research and Evaluation Methods for Time-Sensitive Knowledge Needs
What rapid research and evaluation methods are, where they came from, which techniques save time, what studies comparing rapid and conventional methods have found, and the practices that keep fast research reliable.
Some questions cannot wait for a conventional study. A health service responding to an outbreak needs to know how patients are using its services now, not in two years. A government department reviewing a program before a budget deadline needs evidence within weeks. A business deciding whether to change a product before a seasonal launch has a fixed window. Rapid research and evaluation methods have developed to meet such needs. They shorten the time from question to answer by planning, collecting and analysing data in different ways, while trying to keep enough rigour for the findings to be trusted. This article describes the main rapid approaches, what the evidence shows about how their results compare with slower methods, and the practices that make rapid work reliable.
What "rapid" means
There is no agreed definition of rapid research. Reviews of the field show wide variation. In a systematic review of rapid qualitative research in health emergencies, Johnson and Vindrola-Padros (2017) found studies lasting from 4 days to 1 month. Tricco et al. (2015), in a scoping review of rapid evidence reviews, found that they were completed in 1 to 12 months. Vindrola-Padros et al. (2021) reviewed rapid evaluations in health care and found durations ranging from 6 days to 3 years, with some longer evaluations organised as repeated short cycles of data collection and feedback.
What these studies share is not a fixed length but a design that is shaped by the timing of a decision. McNall and Foster-Fishman (2007) describe rapid evaluation, assessment and appraisal methods as approaches intended to produce timely, useful information for decision-makers, typically by using teams, combining several data sources and analysing data while fieldwork is still under way.
Origins
Rapid methods have a long history in international development and public health. In the 1970s and 1980s, development agencies found that conventional surveys were too slow and expensive to inform project decisions, and developed rapid rural appraisal techniques that relied on short field visits, local informants and teams of researchers from different disciplines. McNall and Foster-Fishman (2007) trace the later development of these into participatory appraisal, rapid assessment procedures and rapid evaluation methods.
The World Health Organization developed rapid evaluation methods to assess the performance of health services. Anker et al. (1993) describe an approach that combined surveys, observation of health facilities and interviews to produce findings within a few weeks for managers. Beebe (2014) set out the Rapid Assessment Process, later called rapid qualitative inquiry, which depends on a team of at least two researchers, intensive interaction with local participants, triangulation of data sources and iterative analysis in which preliminary findings guide further data collection.
Main types of rapid approach
Rapid qualitative research. Interviews, focus groups and observation are carried out over a short period, often by a team, with analysis beginning during fieldwork. Johnson and Vindrola-Padros (2017) found that rapid qualitative studies in health emergencies were used to identify the causes of outbreaks, assess infrastructure and control strategies, and understand how people were using health services. They also highlighted context-specific issues that needed local action and organisational problems in planning and carrying out the response.
Rapid ethnography. Observation within an organisation or community is compressed into a short, focused period. Vindrola-Padros and Vindrola-Padros (2018) reviewed 26 studies using rapid ethnography in health care and found that the approach was increasingly used and could produce findings that informed practice, but that terminology was inconsistent and methods, especially analysis, were often poorly reported.
Rapid evaluation. Programs are evaluated within a time frame set by decision-makers, often with interim findings shared as the evaluation proceeds. Vindrola-Padros et al. (2021) found that rapid evaluators increasingly used repeated feedback cycles and carried out data collection and analysis in parallel.
Rapid evidence reviews. These synthesise existing research using streamlined versions of systematic review methods. Khangura et al. (2012) describe an evidence summary approach developed to answer health system questions within weeks. Hamel et al. (2021), reviewing definitions of rapid reviews, found that the most common defining features were the use of accelerated or streamlined systematic review methods and completion in a shorter time.
Techniques that save time
Rapid research saves time at specific stages. Vindrola-Padros and Johnson (2020) reviewed rapid techniques in qualitative research and found that most focused on reducing or removing the time needed to produce verbatim transcripts. Figure 1 groups common techniques by research stage.
Design. Rapid studies narrow their questions to what the decision requires and often use an existing framework to structure data collection, so that analysis categories are set in advance.
Data collection. Teams split fieldwork, hold daily debriefs and adjust interview guides as findings emerge. Notes may be taken by a second researcher during interviews instead of relying on full transcripts.
Analysis. A widely used technique is rapid qualitative analysis, in which each interview is summarised in a structured template under predefined headings and the summaries are combined into a matrix that compares participants or sites. Hamilton and Finley (2019) describe this approach as it is used in implementation research. Other techniques include coding directly from audio recordings and using tables and spreadsheets to reduce data quickly (Vindrola-Padros & Johnson, 2020).
Reporting. Findings are shared as interim briefings and short reports, so that decision-makers can act before the final report is written.
How do rapid results compare?
The key question is whether faster methods produce different answers. Several studies have compared rapid and conventional methods on the same data.
Qualitative analysis. Taylor et al. (2018) had two teams analyse the same evaluation data, one using rapid analysis and one using full thematic analysis with the framework method. Rapid analysis findings matched 79% of the thematic analysis findings, and thematic analysis findings matched 63% of the rapid analysis findings. Recommendations overlapped less, at 55% and 59%. The time saved was more modest than expected: rapid analysis took much less time to manage the data, but considerably longer to interpret and write up.
Gale et al. (2019) compared a rapid analysis with an in-depth, line-by-line coding analysis of interviews from a process evaluation in the U.S. Veterans Health Administration. The two produced consistent findings, and the rapid analysis delivered actionable results to the program in time to inform implementation. The in-depth analysis took 69 days longer. The authors noted trade-offs, including greater difficulty in rating the strength of constructs.
Vindrola-Padros and Johnson (2020) found that only a minority of the studies proposing rapid qualitative techniques had compared them with conventional methods, and that those comparisons generally found similar results, provided the researchers were experienced.
Evidence reviews. Marshall et al. (2019) tested how rapid review shortcuts would have changed the results of 2,512 Cochrane systematic reviews. Searching only one major database was the least risky shortcut, but about 10% of reviews still showed a moderate or larger change in the pooled result. Other shortcuts, such as using only the largest trial, changed results more often. The authors found no evidence that the shortcuts introduced systematic bias, and concluded that rapid methods may be acceptable when decision-makers can tolerate a known risk that the answer will differ from a full review. Tricco et al. (2015) found that of four case studies comparing rapid and full reviews, three reached similar conclusions.
Limitations and risks
Rapid methods have consistent weaknesses. Johnson and Vindrola-Padros (2017) found that rapid qualitative studies in emergencies reported low-quality data, small samples and too little time to cross-check information. Rapid studies may depend on whoever is easiest to reach, missing harder-to-reach groups whose experience may differ. Compressed analysis may favour obvious themes over subtle ones. Taylor et al. (2018) did not recommend rapid analysis for detailed exploration of complex individual experiences, suggesting it could instead identify issues for further in-depth study.
Rapid work also places heavy demands on researchers. Vindrola-Padros et al. (2020), reflecting on rapid qualitative studies during the COVID-19 pandemic, discussed the need to build teams quickly, obtain ethical approval in short time frames, analyse data in real time and share findings that could be acted on, all under considerable pressure.
When to choose a rapid approach
Rapid methods are not simply a cheaper version of conventional research. They suit some questions better than others. The studies reviewed above suggest four conditions under which a rapid design is a reasonable choice.
The decision has a fixed date. If a budget, contract or launch decision will be made by a known date, a full study that reports afterwards has little value. Gale et al. (2019) chose rapid analysis because the program needed findings while implementation was still under way.
The question is practical and focused. Questions such as "What are the main barriers staff face in using the new system?" suit rapid methods. Questions about subtle or complex individual experiences are less suited (Taylor et al., 2018).
A known risk of error is acceptable. Marshall et al. (2019) frame the choice in terms of risk: a rapid review may be appropriate when decision-makers accept a small but real chance that the answer would differ from a full review. Clinical guidelines and regulatory decisions, they note, still call for comprehensive methods.
Findings can be revisited. Where a decision can be adjusted later, rapid findings can guide the first step and be checked as more evidence becomes available. Vindrola-Padros et al. (2021) found that some rapid evaluations were organised as repeated feedback cycles for this reason.
Where none of these conditions holds, the time saved by a rapid design is unlikely to justify the loss of depth.
Making rapid research reliable
The evidence points to several practices that protect the quality of rapid work:
- Agree the decision and deadline first. Define what decision the research will inform and when the answer is needed, then design backwards from that date.
- Narrow the question. A focused question answered well is more useful than a broad one answered superficially.
- Use experienced researchers. Rapid techniques depend on judgement, and the comparisons that found similar results to full methods generally involved skilled teams (Vindrola-Padros & Johnson, 2020).
- Work in teams and triangulate. Two or more researchers and several data sources reduce the risk of individual bias (Beebe, 2014).
- Use structured templates. Predefined summary headings make rapid analysis consistent and traceable (Hamilton & Finley, 2019).
- Report the shortcuts. State which steps were abbreviated, as Cochrane guidance recommends for rapid reviews (Garritty et al., 2021), so that users can judge how much confidence to place in the findings.
- Plan follow-up. Where the stakes are high, treat rapid findings as interim and confirm them with fuller work later.
Rapid methods in business research
Commercial research has always worked to short deadlines, and many of the techniques described here are already familiar in market research: small qualitative samples, structured note-taking instead of full transcripts, topline reports before final analysis, and iterative rounds of research. The rapid research literature adds two things. First, it provides evidence that, under the right conditions, rapid qualitative analysis produces findings broadly consistent with slower methods (Gale et al., 2019; Taylor et al., 2018). Second, it identifies the conditions under which rapid methods are likely to fail: inexperienced teams, unreachable groups, complex questions and unreported shortcuts. Businesses commissioning fast research can use these findings to decide when speed is acceptable and to ask what was cut to achieve it.
Rapid research is a trade-off between timeliness and completeness. The evidence suggests that the trade-off is often smaller than feared, provided the question is focused, the team is skilled and the shortcuts are reported openly.
References
Anker, M., Guidotti, R. J., Orzeszyna, S., Sapirie, S. A., & Thuriaux, M. C. (1993). Rapid evaluation methods (REM) of health services performance: Methodological observations. Bulletin of the World Health Organization, 71(1), 15–21.
Beebe, J. (2014). Rapid qualitative inquiry: A field guide to team-based assessment (2nd ed.). Rowman & Littlefield.
Gale, R. C., Wu, J., Erhardt, T., Bounthavong, M., Reardon, C. M., Damschroder, L. J., & Midboe, A. M. (2019). Comparison of rapid vs in-depth qualitative analytic methods from a process evaluation of academic detailing in the Veterans Health Administration. Implementation Science, 14, Article 11. https://doi.org/10.1186/s13012-019-0853-y
Garritty, C., Gartlehner, G., Nussbaumer-Streit, B., King, V. J., Hamel, C., Kamel, C., Affengruber, L., & Stevens, A. (2021). Cochrane Rapid Reviews Methods Group offers evidence-informed guidance to conduct rapid reviews. Journal of Clinical Epidemiology, 130, 13–22. https://doi.org/10.1016/j.jclinepi.2020.10.007
Hamel, C., Michaud, A., Thuku, M., Skidmore, B., Stevens, A., Nussbaumer-Streit, B., & Garritty, C. (2021). Defining rapid reviews: A systematic scoping review and thematic analysis of definitions and defining characteristics of rapid reviews. Journal of Clinical Epidemiology, 129, 74–85.
Hamilton, A. B., & Finley, E. P. (2019). Qualitative methods in implementation research: An introduction. Psychiatry Research, 280, Article 112516.
Johnson, G. A., & Vindrola-Padros, C. (2017). Rapid qualitative research methods during complex health emergencies: A systematic review of the literature. Social Science & Medicine, 189, 63–75. https://doi.org/10.1016/j.socscimed.2017.07.029
Khangura, S., Konnyu, K., Cushman, R., Grimshaw, J., & Moher, D. (2012). Evidence summaries: The evolution of a rapid review approach. Systematic Reviews, 1, Article 10. https://doi.org/10.1186/2046-4053-1-10
Marshall, I. J., Marshall, R., Wallace, B. C., Brassey, J., & Thomas, J. (2019). Rapid reviews may produce different results to systematic reviews: A meta-epidemiological study. Journal of Clinical Epidemiology, 109, 30–41. https://doi.org/10.1016/j.jclinepi.2018.12.015
McNall, M., & Foster-Fishman, P. G. (2007). Methods of rapid evaluation, assessment, and appraisal. American Journal of Evaluation, 28(2), 151–168.
Taylor, B., Henshall, C., Kenyon, S., Litchfield, I., & Greenfield, S. (2018). Can rapid approaches to qualitative analysis deliver timely, valid findings to clinical leaders? A mixed methods study comparing rapid and thematic analysis. BMJ Open, 8(10), Article e019993. https://doi.org/10.1136/bmjopen-2017-019993
Tricco, A. C., Antony, J., Zarin, W., Strifler, L., Ghassemi, M., Ivory, J., Perrier, L., Hutton, B., Moher, D., & Straus, S. E. (2015). A scoping review of rapid review methods. BMC Medicine, 13, Article 224. https://doi.org/10.1186/s12916-015-0465-6
Vindrola-Padros, C., Brage, E., & Johnson, G. A. (2021). Rapid, responsive, and relevant? A systematic review of rapid evaluations in health care. American Journal of Evaluation, 42(1), 13–27.
Vindrola-Padros, C., Chisnall, G., Cooper, S., Dowrick, A., Djellouli, N., Symmons, S. M., Martin, S., Singleton, G., Vanderslott, S., Vera, N., & Johnson, G. A. (2020). Carrying out rapid qualitative research during a pandemic: Emerging lessons from COVID-19. Qualitative Health Research, 30(14), 2192–2204. https://doi.org/10.1177/1049732320951526
Vindrola-Padros, C., & Johnson, G. A. (2020). Rapid techniques in qualitative research: A critical review of the literature. Qualitative Health Research, 30(10), 1596–1604. https://doi.org/10.1177/1049732320921835
Vindrola-Padros, C., & Vindrola-Padros, B. (2018). Quick and dirty? A systematic review of the use of rapid ethnographies in healthcare organisation and delivery. BMJ Quality & Safety, 27(4), 321–330.