Appraisal of: "Huang Y, Yang Z, Wang J, Zhuo L, Li Z, Zhan S. (2016). Performance of search strategies to retrieve systematic reviews of diagnostic test accuracy from the Cochrane Library. J Evid Based Med. 9: 77-83. Erratum in: J Evid Based Med 2017;10"

Reviewer(s): 
Caroline Higgins
Corinne Holubowich
Julie Glanville
Full Reference: 
Performance of search strategies to retrieve systematic reviews of diagnostic test accuracy from the Cochrane Library.
Short description: 

The objective of this study was to assess the performance of nine diagnostic accuracy filters in retrieving systematic reviews (SRs) of diagnostic accuracy in Cochrane Library databases, specifically Cochrane Database of Systematic Reviews (CDSR) and the Database of Abstracts of Reviews of Effects (DARE).

Study authors adapted SIGN’s diagnostic studies filter (http://www.sign.ac.uk/search-filters.html), adding concepts of “accuracy measures” and “diagnosis” to create nine strategies. The reference set used to test filter performance in both CDSR and DARE was created using a strategy that combined all the terms used in the filters. Study authors also identified additional diagnostic accuracy studies to include by reviewing each record in CDSR under the topic of diagnosis.

 Overall, filters containing the concept of “accuracy measures” achieved similar sensitivities as those containing “diagnosis” but demonstrated higher precision. Filters containing both concepts demonstrated the highest sensitivity.    

Limitations stated by the author(s): 
  • The use of a highly sensitive strategy, which combined all terms being tested, to create the reference set likely overestimated the performance of other strategies and may have missed some relevant articles which did not contain the terms used in filters.
  • The focus of the study was the retrieval of systematic reviews of diagnostic test accuracy and not retrieval of primary studies. Authors acknowledged methodological filters are primarily designed to retrieve original studies.
Limitations stated by the reviewer(s): 
Although the paper was published in 2016, the authors do not share with readers that although DARE is archived and can still be searched, it is no longer being updated (searches for systematic reviews ceased at the end of 2014; structured abstracts were published on DARE until March 31, 2015); The authors acknowledged that using diagnostic accuracy filters designed to retrieve primary studies is a limitation when compiling the set of relevant reviews. This means that this study’s findings are not necessarily generalizable to filter performances for the retrieval of primary diagnostic accuracy studies.
Study Type: 
Single study
Related Chapters: 
Diagnostic accuracy
Supplemental publications to the study: 
ERRATUM.
Tags:
  • Share this