Evaluating Health Services: A Reporter Covers the Science of Research Synthesis

TitleEvaluating Health Services: A Reporter Covers the Science of Research Synthesis
Publication TypeReport
Year of Publication2004
AuthorsMoynihan R
Date Published65 p.
InstitutionMilbank Memorial Fund
CityNew York
ISBN Number1-887748-56-3
KeywordsEvaluation Studies [Publication Type]; Health Services
AbstractThe systematic review is now widely regarded as the least biased and most rational way to summarize the research evidence that evaluates health care interventions meant to prevent and treat illness. A systematic review can help distinguish therapies or interventions that work from those that are useless, harmful, or wasteful. It can reliably estimate how well different options work, and it can identify gaps in knowledge requiring further research. Efforts to synthesize research evidence date back centuries, but the science of the systematic review has been greatly refined in the past few decades, particularly in the social and health sciences. As Richard Light and David Pillemer explain in their groundbreaking work Summing Up: The Science of Reviewing Research, "Without a clear picture of where things stand now, simply adding one new study to the existing morass is unlikely to be very useful…. For science to be cumulative, an intermediate step between past and future research is necessary: synthesis of existing evidence." 1 The basic steps of a systematic review include formulating a question; finding relevant studies; selecting and assessing the studies; summarizing and synthesizing study results; interpreting the review results; and maintaining and updating the review. Systematic reviews are being produced by many organizations in the United States and around the world, and they are being used across much of the health care landscape. Growing numbers of nurses, pharmacists, doctors and other health professionals, patients, insurers, policymakers, advocates, health care executives, and journalists are looking to systematic reviews of the evidence to inform their thinking, decision making, and practice. But systematic reviews have significant limitations as well as benefits. The studies being reviewed are often incomplete, deficient, or skewed toward the most profitable treatments. Sometimes systematic reviews themselves are poorly conducted, as for example when the search for relevant studies has not been as comprehensive as possible. Often systematic reviews will conclude that there is not enough strong evidence to support or refute a technology that some clinicians and patients consider promising—yet decisions about its use must still be made and defended. Notwithstanding their pitfalls, systematic reviews promise to improve the quality of many health care decisions.