The aim of the study was to assess how easily one could identify systematic reviews of adverse effects in the two major databases of systematic reviews: the Database of Abstracts of Reviews of Effects (DARE) and the Cochrane Database of Systematic Reviews (CDSR).
A gold standard set of records (270 systematic reviews) was identified from DARE and CDSR by carrying out searches for systematic reviews of adverse effects published from 1994 to 2005. The search strategies used a combination of text words in the title and abstract, subject headings (MeSH) and subheadings. In addition, DARE records in progress were hand searched. The search terms used to identify the systematic reviews were assessed for their usefulness in retrieving relevant records by measuring their sensitivity and precision.
The results showed that ‘floating’ subheadings provided the highest sensitivity in both DARE and CDSR (85% and 64% respectively). All single search terms in CDSR resulted in very low precision (0% - 3%). Single MeSH terms provided the highest level of precision in DARE (67%-73%). The most sensitive search strategy in DARE used a combination of text words in the title and abstract, a MeSH term and ‘floating’ subheadings (94%). The precision of this strategy was 16%. In CDSR the most sensitive strategy used the ‘floating’ subheading ‘adverse effects’ and text word search ‘adverse near/20 objectives’ in the abstract field (79%). The precision of this strategy was 3%.
Limitations stated by the reviewer(s):
Even though this study provides the most up-to-date evidence on the topic of identifying systematic reviews of adverse effects in CDSR and DARE, readers should bear in mind that the study was published in 2006. Adverse effects terms may be increasingly prevalent in the title, abstract and indexing terms of records of studies published more recently compared to those published up until 2006. Readers may, therefore, wish to consider the appraisals of more recent work by Golder et al which address identifying single studies reporting adverse effects.
Changes made in the two databases in recent years (such as new user interfaces and search functionality) may also influence the validity of the study findings.
In addition to the limitation outlined above regarding the very low number of systematic reviews of adverse effects in CDSR (14), it should also be noted that reviews of adverse effects make up a relatively low percentage of the total number of reviews in DARE and an even lower percentage of the total number of reviews in CDSR and that this might have some bearing on the findings of the study.
Also, it should be noted that the same set of search terms was used both to identify the systematic reviews for the gold standard set of records, and to measure the performance of single search terms and their combinations. This might have influenced the findings of this study.
This publication is related to Su Golder’s PhD Thesis “Evaluating and Optimising the Retrieval of Research Evidence for Systematic Reviews of Adverse Drug Effects and Adverse Drug Reactions” from 2013. The thesis is available from http://etheses.whiterose.ac.uk/4749/