Appraisal of: Waffenschmidt S, Janzen T, Hausner E, Kaiser T. Simple search techniques in PubMed are potentially suitable for evaluating the completeness of systematic reviews. J Clin Epidemiol. 2013; 66(6):660-665.

Short description: 

The aim was to determine whether PubMed's Related Citations (RelCits) and/or a simple-structured Boolean search (SSBS) are efficient and reliable search techniques to assess the completeness of an evidence base consisting of published randomized controlled trials (RCTs). A retrospective analysis of 330 PubMed citations included as relevant in 19 systematic reviews (SRs) of drugs was undertaken, with the proportion of relevant citations identified by both RelCits and SSBS determined. It was concluded that combining the first 20 RelCits and an SSBS in PubMed is a suitable method to assess the completeness of an evidence base of published RCTs.

Limitations stated by the author(s): 
  • Limited generalizability of results because of the development of a gold standard on the basis of SRs completed by IQWiG.
  • Questionable if the techniques also work with new citations not yet indexed as well as citations without an abstract.
  • Applicability of the findings is currently limited to the search for RCTs; further research is needed to test the applicability to non-RCTs.
  • Only PubMed citations were analyzed.
Limitations stated by the reviewer(s): 
No additional limitations were detected by the reviewers.
Study Type: 
Single study