Appraisal of: Arber M, Garcia S, Veale T, Edwards M, Shaw A, Glanville J. Sensitivity of a search filter designed to identify studies reporting health state utility values [poster].

Reviewer(s): 
Short description: 

The retrieval of studies that report health state utility values (HSUVs) is an important aspect of information retrieval for HTA and economic model production.  The York Health Economic Consortium (YHEC) created and tested three search filters designed to identify studies reporting HSUVs. The relative recall method was used to test the sensitivity of each filter. A quasi-gold standard set of 293 studies was compiled by identifying a sample of systematic reviews of studies reporting HSUVs. All three filters tested retrieved the same 267 citations from the quasi-gold standard (0.91 sensitivity). Filter 3 had the highest precision at 0.003. The 26 studies missed by the filters did not explicitly refer to HSUVs within the citations. Inclusion of the “quality of life” subject heading would have retrieved 20 of those studies, but also increased overall results by 116%. Further analysis of the 26 records missed by the search filters is planned to try to improve sensitivity without a significant loss in precision. The search strategy for the most precise search filter is included in the poster.

Limitations stated by the author(s): 

The retrieval of studies that report health state utility values (HSUVs) is an important aspect of information retrieval for HTA and economic model production.  The York Health Economic Consortium (YHEC) created and tested three search filters designed to identify studies reporting HSUVs. The relative recall method was used to test the sensitivity of each filter. A quasi-gold standard set of 293 studies was compiled by identifying a sample of systematic reviews of studies reporting HSUVs. All three filters tested retrieved the same 267 citations from the quasi-gold standard (0.91 sensitivity). Filter 3 had the highest precision at 0.003. The 26 studies missed by the filters did not explicitly refer to HSUVs within the citations. Inclusion of the “quality of life” subject heading would have retrieved 20 of those studies, but also increased overall results by 116%. Further analysis of the 26 records missed by the search filters is planned to try to improve sensitivity without a significant loss in precision. The search strategy for the most precise search filter is included in the poster.

Limitations stated by the reviewer(s): 
Ovid MEDLINE was the only database searched for this study.
Study Type: 
Single study