Search filters

Scope: use this with methods papers on search filter design, comparison of search filter performance

Appraisal of: McKinlay RJ, Wilczynski NL, Haynes RB, Hedges Team. Optimal search strategies for detecting cost and economic studies in EMBASE. BMC Health Serv Res 2006;6:67.

Short description: 
This study reported the development of a search strategy for cost and economics studies in Embase. The gold standard was created by handsearching 55 journals for the year 2000. The authors provide strategies to find studies about costs and about economic studies broadly defined. A cost strategy with a sensitivity of 100% and precision of 8.6% is presented along with an economic studies strategy of 100% with 1.4 % precision. Best compromise strategies are offered for costs (sensitivity 98.4% and precision 18.2%) and economics (sensitivity 96.8% and precision 4.3%)
Limitations stated by the author(s): 
Precision will be lower when searching the entire Embase database. Multivariate statistical techniques might yield better performing strategies but authors did test a logistic regression approach.
Limitations stated by the reviewer(s): 
55 journals were chosen for frequency of yield and these may be different to the wider range of journals which publish cost and economics studies. The handsearched records are for one year only. High impact factor journal were chosen and these may be different to other journals perhaps in terms of encouraging better study reporting. The authors have a broad definition of economics which means their strategy may not be optimized to find economic evaluations.
Study Type: 
Single study
Related Chapters: 

Appraisal of: Glanville J, Kaunelis D, Mensinkai S. How well do search filters perform in identifying economic evaluations in MEDLINE and EMBASE. Int J Technol Assess Health Care 2009;25(4):522-529.

Short description: 
This research was undertaken to develop search filters to identify economic evaluations in MEDLINE and Embase. The performance of these filters and published filters were tested in terms of sensitivity and precision using a gold standard set of known economic evaluations. Thirteen filters for MEDLINE were tested and three yielded a sensitivity of 99%. The best precision for one of these filters was 4%. The best compromise filter provided 84% sensitivity for 13% precision. Four filters provided greater than 99% sensitivity in Embase with precision ranging from 1.5- 2.9%. The best compromise filter provided 23% precision and 63% sensitivity.
Limitations stated by the author(s): 
The study relies on a gold standard obtained from NHS EED and therefore the NHS EED filter may overperform when tested against it and also how far NHS EED represents typical economic evaluations needs to be judged. It was not possible to assess whether the search filters identified economic evaluations not captured by NHS EED. The performance of filters for finding MEDLINE in process or PubMed records was not tested. Excluding animal studies improved precision in MEDLINE but not in Embase.
Limitations stated by the reviewer(s): 
No additional limitations detected by the reviewer.
Study Type: 
Single study
Related Chapters: 

Appraisal of: Golder S, Loke YK. Sensitivity and precision of adverse effects search filters in MEDLINE and EMBASE: a case study of fractures with thiazolidinediones. Health Info Libr J 2012;29(1):28-38.

Short description: 
A systematic review of fracture-related adverse effects associated with the use of thiazolidinediones was used as a case study to calculate the sensitivity, precision and Number Needed to Read (NNR) of published adverse effects search filters in MEDLINE and Embase. 12 MEDLINE filters and 3 Embase filters were tested. The results showed that 4 search filters in MEDLINE achieved high levels of sensitivity (95 or 100%) with improved levels of precision compared with searches without any adverse effects filters. The highest level of precision in MEDLINE (55%) was achieved with search filters that relied only on subject headings (MeSH). No search filter in Embase achieved sensitivity higher than 83% and precision remained low using any of the filters (all under 5%).
Limitations stated by the author(s): 
The main limitation to this study is that because only one systematic review was used as a case study, the generalisability of the study results is limited. In addition, this case study is of a particular named adverse effect (fractures), while a case study of a safety profile systematic review, in which all adverse effects are searched for, might have given different results. Another limitation is the adaptations made to some of the tested search filters. These filters were originally created for use in searches where the adverse effects are not known in advance of searching, while in this case study, these filters were used in addition to search terms for a named adverse effect (fracture terms).
Limitations stated by the reviewer(s): 
In addition to the limitation noted above by the authors, it should be noted that the number of included studies on which the search filters were tested was relatively small (19 records in MEDLINE and 24 records in Embase). It should also be noted that the filters were tested as published. Testing of individual terms within the filters might have resulted in higher precision and better trade-off between sensitivity and precision than the full filters.
Study Type: 
Single study
Related Chapters: 

Comments from the authors:

Appraisal of: Golder S, Loke Y. The performance of adverse effects search filters in MEDLINE and EMBASE. Health Info Libr J 2012;29(2):141-151.

Short description: 
The aim of the study was to measure the sensitivity of published adverse effects search filters in MEDLINE and Embase. The performance of all the individual search terms included in the filters was also measured. The results showed that 93% of the MEDLINE records and 97% of the Embase records could be retrieved by using adverse effects search filters. High sensitivity could also be achieved by using individual adverse effects ‘floating’ subheadings such as Adverse Drug Reaction (83%) or Side Effect (83%) in Embase. In MEDLINE the highest sensitivity using individual adverse effects ‘floating’ subheadings was achieved by using Adverse Effects (51%). Some free text terms for adverse effects in the title and / or abstract were also found to be useful, but the authors recommend applying these terms only concurrently with other search terms such as subheadings. The sensitivity of the few existing indexing terms (MeSH and Emtree) for adverse effects was shown to be low.
Limitations stated by the author(s): 
The main limitation of this study was that the precision of search terms and search filters was not measured. The most sensitive search filters or search terms might be those with the lowest precision and might therefore result in very low precision (that is, the retrieval of very large numbers of irrelevant records). Further research that measures the precision of these search terms and filters is required in order to determine the full value and effectiveness of using adverse effects terms in search strategies. Caution should be applied when using the results of this study, especially as some of the search terms with relatively high sensitivity appear to have only a vague connection to adverse effects (such as ‘Therapeutic Use’ and ‘Pharmacology’).
Limitations stated by the reviewer(s): 
No additional limitations detected by the reviewer
Study Type: 
Single study
Related Chapters: 

Comments from the authors:

Syndicate content