Systematic reviews

Scope: this is a review carried out with extensive searches, clear inclusion and exclusion criteria and a qualitative or quantitative summary of the results of the review. This index term can be applied to descriptions of systematic reviews, collections of systematic reviews, issues of identifying systematic reviews and their value to the HTA process

Appraisal of: "Yao X, Vella E, Brouwers M. How to conduct a high-quality systematic review on diagnostic research topics. Surg Oncol. 2018 Mar;27(1):70-75."

Reviewer(s): 
Short description: 

Authors suggest six steps are necessary for conducting high-quality systematic reviews on diagnostic topics around the objective, research question, searching for literature, assessing risk of bias, analyzing, and interpreting data. The review aims to instruct researchers on diagnostic methodology and recent updates in the field.

Rather than designing a study around the elements of PICO, PIRO is recommended in which “I” represents index test, and “R” represents reference standard.

The literature search for diagnostic topics can begin with using an existing systematic review that answers the research question, and is of satisfactory quality. The search date of that study can be used as a starting point to search forward in MEDLINE, Embase, Cochrane Database of Systematic Reviews, and PROSPERO. A grey literature search is also recommended.

When no appropriate systematic reviews are available, a search of primary studies will need to be conducted. Databases to be used in this case include MEDLINE, Embase, and Cochrane Library databases, as well as related conference proceedings. It is preferred that language limits are not used and that a grey literature search is carried out if deemed appropriate by researchers. The search should include Medical Subject Headings terms and keywords representing the diagnostic test and target condition. Adding terms to capture diagnostic accuracy or study design is not recommended, as indexing of this concept has not been proven to be dependable.

This is a second article in a series. First article: Yao X, Vella E. How to conduct a high-quality original study on a diagnostic research topic. Surg Oncol. 2017;26(3):305-9.

 

Limitations stated by the author(s): 

Authors noted that in order to keep the article brief, they did not fully explore the concepts of how to design strategies for different databases or peer review.

Limitations stated by the reviewer(s): 
Authors made general observations about the quality of diagnostic studies needing improvement without expanding on any evidence to support the claim.
Study Type: 
Review

Appraisal of: “Korevaar DA, Hooft L, Askie LM, Barbour V, Faure H, Gatsonis CA, et al. Facilitating Prospective Registration of Diagnostic Accuracy Studies: A STARD Initiative. Clin Chem. 2017;63: 1331–1341.”

Short description: 

This paper provides the arguments for prospective registration of diagnostic test accuracy (DTA) studies in public databases. The authors do not advocate the creation of a specific registry for DTA studies, but registration is recommended in any available known public registry.  A list of possible registries is provided in Table 1 of the paper.

The authors propose a modified version of the WHO trial registration data set (Table 2 in the paper) which specifies the fields that would be needed for registry records to capture the key elements of a DTA study. 

Using their modified version of the WHO trial registration data set, the authors evaluated the reported information for the 30 most recently registered DTA studies in the WHO ICTRP in November 2016.  This small analysis provides some information for searchers who are searching for DTA studies in ICTRP.   In the sample of thirty studies, 90% reported that the study was a DTA study in the title of the record, 97% reported the target condition and 100% reported the index test.

The authors call on trial registries to ensure informative data are included in registry records to enhance identification.  They also call on journal editors to encourage DTA study registration prior to publication.

Limitations stated by the author(s): 

This is a position paper and it does not dwell upon limitations of its methods.

Limitations stated by the reviewer(s): 
The information for searchers is largely from the analysis of a small sample of thirty recent study records in ICTRP, but this information is highly valuable in indicating which aspects of DTA studies might be identifiable and retrievable within that database.

Appraisal of: "Preston L, Carroll C, Gardois P, Paisley S, Kaltenthaler E. Improving search efficiency for systematic reviews of diagnostic test accuracy: an exploratory study to assess the viability of limiting to MEDLINE, EMBASE and reference checking."

Short description: 

This exploratory study analysed a convenience sample of nine Health Technology Assessments (HTA) reviews of diagnostic test accuracy (DTA) to identify which of the included citations were indexed on MEDLINE or EMBASE; to identify the number and proportion of citations that were retrieved by the search strategies with or without the addition of checking the reference lists of included studies.                     

Of the 302 included citations included in the nine reviews, 287 (95%) were indexed in MEDLINE and EMBASE. The reviews searches accounted for 256 (85%) of the included citations and a further 24 (8%) could be identified from the reference lists of included citations. 7% of the citations (22/302) were not found by searching or reference checking.

The proportion of citations identified using both searches of MEDLINE, and EMBASE and checking the reference lists resulted in the identification of 280/302 (93%) included studies. The authors suggest that there might be a case for restricting systematic review searches  of DTA studies to MEDLINE, EMBASE and the reference lists of included studies. 

Limitations stated by the author(s): 

The study’s pragmatic, convenience sample of a small number of published DTA reviews was intended as an exploratory analysis only. The  authors did not conduct an analysis to assess the impact of excluding the unidentified studies on the conclusions of the HTAs, so the results of the study are uncertain. 

Limitations stated by the reviewer(s): 
Six of the nine studies in the sample used a published filter for diagnostic studies, which is not recommended by Cochrane. This may have impacted the sensitivity of the searches and the sample not be representative of other DTA reviews.
Study Type: 
Single study
Related Chapters: 

Appraisal of: Mathes T, Walgenbach M, Antoine SL, Pieper D, Eikermann M. Methods for systematic reviews of health economic evaluations: a systematic review, comparison, and synthesis of method literature. Med Decis Making. 2014 Oct;34(7):826-40.

Short description: 

The authors compare existing methods and suggest best practices for the preparation of systematic reviews of health economic evaluations (SR-HE). While most of this review falls outside the realm of information retrieval, one section summarizes the existing methods used in conducting literature searches for SR-HEs. The authors found that many steps in the SR-HE process have unjustifiable divergences, but that the literature search and selection are the steps with the most concordance. Best practices for literature searching include searches of MEDLINE and Embase, plus one health economics database (recommended are NHS EED or HEED). A systematic search strategy should be employed using relevant PICO elements combined with an economic evaluation search filter. Search strategies should be adapted to different databases, especially regarding the indexing and classifications of economic studies within each database. 

Limitations stated by the author(s): 

Since the authors searched specifically for papers that refer to SR-HEs, other relevant papers, such as those that refer to quantifying input values for economic evaluations, may have been missed. The authors note thusly that they could not identify all the relevant literature that could be applicable to SR-HEs. Also, the authors note that their best practice suggestions should only be considered as a framework due to the vague or general descriptions of methods found in some publications and because of discrepancies between the publications included in this study.

Limitations stated by the reviewer(s): 
No additional limitations detected by the reviewers.
Study Type: 
Single study
Syndicate content