Appraisal of: "Harboe I, Desser A, Nordheim L, Glanville J. PD85: Testing search filters to retrieve economic evaluations in Embase. Int J Technol Assess Health Care. 2018;34(S1):158."

Reviewer(s): 
Short description: 

The authors tested a self-created Ovid Embase cost-effectiveness analysis (CEA) search filter, compared to six other published filters designed to identify economic evaluations in Embase. In this filter validation study, sensitivity, precision and specificity were measured against a gold standard set of economic evaluations derived from the National Health Service Economic Evaluation Database (NHS EED). The set was populated with NHS EED citations published in 2008-2013 that were also available in Ovid Embase (n = 2,198). Testing of the CEA filter resulted in a sensitivity of 0.899 and precision of 0.029. No filter achieved the stated objective of achieving a sensitivity of at least 0.90 with a precision of 0.10, and specificity of at least 0.95, although one tested filter was close to achieving that objective. The authors conclude that it is challenging to develop economic search filters that balance sensitivity and precision. Choice of filter should be based on the researcher’s acceptable levels of performance.

Limitations stated by the author(s): 

None stated.

Limitations stated by the reviewer(s): 
The gold standard set of economic evaluations was derived entirely from the NHS EED database which was itself updated by searches done in various databases. The filters used in these search updates would control the contents of both the database and the gold standard set used here, which could potentially have skewed the performance of the tested filters. This is only a conference abstract with limited details of the study methods, results and conclusions. Further details can be found in the lead author’s thesis (see Supplemental publications to the study, below).
Study Type: 
Single study
Related Chapters: 

Based on: Harboe I. Testing the best performing methodological search filters to retrieve health economic evaluations in Embase: a filter validation study [master’s thesis]. [Bergen (Norway)]: Western Norway University of Applied Sciences; 2017 May 15. https://hvlopen.brage.unit.no/hvlopen-xmlui/bitstream/handle/11250/2669232/Masterthesis_Harboe.pdf