Clinical effectiveness

Last revised: 
2019-12-19

Introduction

This domain focuses on the identification of evaluations of the efficacy or effectiveness of a technology (device, medicine, vaccine, procedure or system) or intervention. These evaluations focus on whether a technology works, as well as the magnitude of health benefits or harms caused by the technology. Searching for adverse effects is summarized in the SuRe Info Safety chapter, and diagnostic tests in the SuRe Info Diagnostic Accuracy chapter. Study designs used to assess clinical effectiveness of a technology or intervention include randomized controlled trials (RCTs), quasi-experimental studies, and observational studies. Clinical effectiveness searches will focus on the identification of reports of these study types (1,2).

This chapter is primarily based on Cochrane’s 2019 update of the Searching for and selecting studies chapter (Chapter 4) of the Cochrane Handbook for Systematic Reviews of Interventions (3) and the EUnetHTA guideline on information retrieval for systematic reviews and health technology assessments of clinical effectiveness (4).

This chapter is the result of extensive Cochrane work, and appraisals have not been prepared for studies cited in the Cochrane Handbook.

Sources to search

The choice and number of sources selected to search will depend on the research question, which resources can be accessed, and time and budget constraints (3,5). The Cochrane Handbook states the following databases should be searched as a minimum to identify as many relevant efficacy studies as possible and minimise the risk of publication bias:

While CENTRAL includes records from MEDLINE, Embase, clinical trial registers and other bibliographic databases, the Cochrane Handbook (section 4.3.1.1) recommends supplementary searches of MEDLINE and Embase for comprehensiveness and currency since there is a time delay between records being indexed in MEDLINE/Embase and appearing in CENTRAL.

There are various different interfaces for these databases, some free and some fee-paying, check your own institution library to see which resources you can access. For more information see the SuRe Info Service providers and search interfaces page.

Subject specific databases

The 2019 Cochrane Handbook (section 4.3.1.4) states that it is highly desirable to search appropriate subject-specific databases (3). There is mixed evidence on whether searching subject-specific databases adds value in terms of identifying additional unique references, and the decision about whether to search subject specific databases may depend on the research question or topic area (6–11).

Examples of subject-specific bibliographic databases are listed in an Appendix. (Further information will also be available in the Technical Supplement associated with the Cochrane Handbook.)

National/regional databases

The 2019 Cochrane Handbook (section 4.3.1.4) recommends searching national and regional databases (3), as these may index journals not included in international bibliographic databases such as MEDLINE and Embase, and could minimize the risk of language bias (12–14). There may be particular relevance in searching regional databases for certain topic areas, for example, searching Chinese databases may identify additional trials in searches on Chinese traditional medicine (15). Examples of regional and national bibliographic databases are listed in the Appendix.  (Further information will also be available in the Technical Supplement associated with the Cochrane Handbook.)

Bethel and Rogers (2019) recommend using search summary tables to evaluate the impact of decisions regarding bibliographic database selection (5). Search summary tables make it easier to identify the resources that contribute unique records and may aid decision-making for update searches.

Clinical trial registers, ongoing studies and unpublished data

Sources of ongoing and unpublished studies should be included in the search. Searching for such studies can prove challenging, but should be undertaken to minimise bias (16). Ongoing studies can be found in trials registries, whose records are updated when the trials are completed and published. 

The following is a selected list of trial registries and a search engine:

Some clinical trials registries are region or topic specific (for a more exhaustive list see Chapter 4 of the 2019 Cochrane Handbook (3) or the YHEC clinical trials website).

Investigators’ documentation and clinical study reports are another source of unpublished data so contact with principle investigators or researchers may be tried. Isojärvi et al 2018 have summarised research evidence around identifying unpublished data (17) and research highlights the need to search multiple trials registers (18).

Supplementary searching

Consider which supplementary search methods to use; sources will depend on the topic of the search (2019 Cochrane Handbook section 4.3.5). These methods may bring up publications that were not found in the original search or identify other concepts that have been missed. They may also identify comments, errata, retractions or related studies (19–21)

There is evidence that suggests hand searching may identify unique studies not identified through bibliographic database searches, particularly in non-English language journals (22). However, the value of hand searching may vary across subject areas (23).

Citation index searching can help identify studies particularly where subject searches are challenging (24, 25), and the 2019 Cochrane Handbook (section 4.3.5) recommends screening previous reviews on the same topic to identify studies (3). For further information on citation searching, see the SuRe Info section on the Value of using different search approaches.

Web-searching may identify individual studies or organisations such as academic units or research bodies that publish relevant material. The usefulness of this is also topic dependent and further information will be available in the Technical Supplement associated with the Cochrane Handbook.

Searching for grey literature (or literature not controlled by traditional commercial publishing including reports, dissertations, theses, databases of conference abstracts) is considered highly desirable for Cochrane Reviews (Cochrane Handbook section 4.3.5). Sources of grey literature are listed in the Appendix. The current Cochrane Handbook states that searching for grey literature is highly desirable (3), however, there is discussion regarding the value of grey literature for certain topics, so the time and effort spent searching grey literature sources should be carefully considered when planning the search (24 ,26).

Regulatory agency, manufacturer websites and clinical study reports may be useful for providing more extensive detail on interventions than is available in journal articles (17). Regulatory agencies provide access to detailed pharmaceutical submission documents such as the EMA’s public assessment reports (EPARs), FDA drug approval package or DAPs, as well as product recalls, market withdrawals or safety alerts. 

Health technology assessments (HTAs) published by national health technology assessment agencies can provide detailed information on the clinical effectiveness, economic analysis and patient related issues around new health technologies.

Designing search strategies

The 2019 Cochrane Handbook (section 4.4) provides guidance on issues to consider when designing search strategies for systematic reviews (3). The PICO model (Patient or Population or Problem; Intervention; Comparison; Outcome) is commonly used to develop the structure of a search strategy for clinical effectiveness research questions, but other models are available (27).

Some systematic reviews may include non-randomized controlled trials (2019 Cochrane Handbook Chapter 24) (3). Search strategies for these study types can be problematic as they are not well defined or indexed consistently (28).

For further information, see the SuRe Info chapter on Search strategy development.

Search filters

Search filters are combinations of search terms designed to retrieve particular types of references, including specific methodological study designs. The 2019 Cochrane Handbook (section 4.4.7) recommends using specially designed and tested filters (such as the Cochrane Highly Sensitive Search Strategies for identifying randomized trials in Ovid MEDLINE) only when appropriate (3). A search filter should not be used in a pre-filtered database such as CENTRAL.

For further information on the sources of methodological filters (including filters to identify randomized controlled trials and observational studies), see the SuRe Info section on Search filters.

Documenting searches

For further information regarding documenting and reporting searches, refer to the SuRe Info chapter.

Reference list


 

 

AttachmentSize
ClinicalEffectivenessChapter_Appendix.pdf138.08 KB