Economics

Scope: this includes cost-effectiveness studies, costs and costing studies and utilities

Appraisal of: "Dakin H, Abel L, Burns R, Yang Y. Review and critical appraisal of studies mapping from quality of life or clinical measures to EQ-5D: an online database and application of the MAPS statement. Health Qual Life Outcomes. 2018;16:202."

Short description: 

This paper presents an updated systematic review of studies reporting mapping algorithms from clinical or patient reported outcome measures to EQ-5D-3L or EQ-5D-5L, as well as an assessment of how far mapping studies conform to the MAPS reporting statement.

The review identified 144 mapping studies reporting 190 algorithms mapping from 110 different instruments onto EQ-5D. The 15/17 studies published in 2016 had low adherence to the MAPS checklist.

The mapping studies can be accessed in the Health Economics Research Centre (HERC) Database of Mapping Studies, available at https://www.herc.ox.ac.uk/downloads/herc-database-of-mapping-studies. This database was created in 2013 and is focused on studies mapping to EQ-5D.

Limitations stated by the author(s): 

The review did not include a search of EMBASE. Single reviewer data extraction was undertaken. Only 2016 studies were assessed against the MAPS criteria.

Limitations stated by the reviewer(s): 
The authors do not describe how frequently they update the database.
Study Type: 
Single study

Appraisal of: "Arber M, Wood H, Isojarvi J, Glanville J. Which information sources should be used to identify studies for systematic reviews of economic evaluations in healthcare? Value Health. 2017 Oct/Nov;20(9):A738. Abstract PRM46."

Reviewer(s): 
Short description: 

NHS EED and HEED, two key databases for retrieving health economics information, have closed. Based on this changed landscape, this abstract assesses which database are now the best sources of information for retrieving economic evaluations for models and systematic reviews. The authors built a quasi-gold standard database of 351 records compiled from 46 systematic reviews of economic evaluations. Nine databases were searched for each record. Embase had the highest yield (0.89), followed by Scopus (0.84) and MEDLINE and PubMed (both 0.81). The HTA database identified the highest number of unique citations (13/351). All nine database combined retrieved 337/351 (0.96) records. The authors conclude that for most systematic reviews, Embase, the HTA database and either PubMed or MEDLINE are likely sufficient to identify economic evaluations found in bibliographic databases. Searching a multi-disciplinary database may also be useful, especially in non-clinical topics. Supplementary search techniques may be more efficient than searching a larger number of databases.

MEDLINE search strategies reported in source systematic reviews were also assessed. 10/29 (34.5%) of re-run search strategies missed at least one of the included records found in MEDLINE (with 25 citations missed in total). Weaknesses in the population or intervention concepts, rather than the economics concept, were identified as negatively impacting search retrieval.

Limitations stated by the author(s): 

None stated.

Limitations stated by the reviewer(s): 
Since the results were published only in abstract form, very limited information was presented on methods, results and conclusions.
Study Type: 
Single study

Appraisal of: "Arber M, Garcia S, Veale T, Edwards M, Shaw A, Glanville JM. Performance of Ovid MEDLINE search filters to identify health state utility studies. Int J Technol Assess Health Care. 2017 Jan;33(4);472-80."

Reviewer(s): 
Short description: 

The retrieval of studies that report health state utility values (HSUVs) is an important aspect of information retrieval for HTA and economic model production.  This study first assessed three MEDLINE search filters designed by the York Health Economic Consortium (YHEC) to identify studies reporting HSUVs. The relative recall method was used to test the sensitivity of each filter. Three quasi gold standard (QSG) sets of relevant studies were compiled from reviews of studies reporting HSUVs. The first QSG (consisting of 294 records) was used to assess the performance of the three initial filters. The best performing of the three filters was further developed using the second QSG (139 records). Ultimately, three final search filters were validated using the third QSG (139 records). The first final search filter is sensitivity maximizing, with 95% sensitivity and a number needed to read (NNR) of 842. The second filter balances sensitivity and precision, and has a 92% sensitivity with an NNR of 502. The third filter is precision maximizing, with 88% sensitivity and an NNR of 383. Real world volume of retrieved records was also tested to illustrate the impact of using the three filters for three example health conditions. Having a range of sensitivity and precision options allows researchers to choose filters based on their search requirements. The authors believe that these are the first validated filters for retrieving HSURs. Search strategies for all three final filters are presented in the article.

Limitations stated by the author(s): 

The robustness of the relative recall method depends on the quality of the searches within each systematic review. Despite passing a quality checklist, some of the systematic reviews employed less than optimal search methods, which may have negatively affected study identification.

Also, a larger quasi-gold standard set may have improved the generalizability of the study findings. During filter development, several terms (utility loss, disutility, short form, SF-12) were removed to increase precision. However, during final validation, it was found that inclusion of these search terms would have retrieved an additional three records, revealing the difficulty in making filter decisions based on a small number of relevant records. These removed terms could be added to the filters to maximize sensitivity. Similarly, some terms added to the filters resulted from an analysis of a very small number of records. These revisions were made to increase sensitivity, but the missed records retrieved through these additions were outliers, and less likely to be representative of the wider body of relevant results. The addition of these terms may increase search result size, but potentially have minimal impact on the total number of relevant results retrieved.

The initial filters created by YHEC were developed through pragmatic and traditional subjective methods, rather than using objective filter design methods. Despite the pragmatic development, these strategies performed well in testing.

The final filters are designed to find reviews, cost-utility analyses and utility elicitation studies that report HSUVs, as well utility mapping studies. Since the filters do not distinguish between these different types of research, precision for research seeking only utility elicitation studies, for example, will be lower than the results reported in the article.

 

 

Limitations stated by the reviewer(s): 
These filters have been validated only for Ovid MEDLINE searches. Translations of these filters to other databases may not produce search results with similar sensitivity or NNR.
Study Type: 
Single study

Appraisal of: "Ara R, Brazier J, Peasgood T, Paisley S. The identification, review and synthesis of health state utility values from the literature. Pharmacoeconomics. 2017;35:43–55."

Short description: 

This paper is a guidance document providing an overview of how health state utility values can be identified, reviewed and synthesised when conducting systematic reviews. The paper includes a case study of a review in osteoporosis-related conditions.

In terms of study identification, the authors note that a range of study designs could be relevant and that a number of instruments could be required including condition-specific preference based measures or generic preference based measures. They recommend that a variety of resources and methods should be used to identify studies. As well as electronic databases, searchers should also look at reference lists, conduct key author and citation searches and contact experts.

The authors recommend caution in using filters too early in the search process, to avoid missing potentially relevant studies. The authors also note the absence of dedicated subject headings within MeSH and EMTREE, and that although general subject headings such as ‘Quality of life’ will yield relevant studies, they are likely to demonstrate poor precision. Free text terms should be included in searches and are categorised as general terms (such as QALY), instrument specific terms (such as EQ-5D) and terms describing methods of utility elicitation such as standard gamble.

Limitations stated by the author(s): 

The authors discuss the issues of searching for utility studies including current limitations. They do not discuss limitations of their review and do not provide details of their methods in this publication.

Limitations stated by the reviewer(s): 
The overview and recommendations in relation to searching do not seem controversial.
Study Type: 
Single study

Appraisal of: Shannon M. Sullivan, Kate Tsiplova & Wendy J. Ungar. A scoping review of pediatric economic evaluation 1980-2014: do trends over time reflect changing priorities in evaluation methods and childhood disease? Expert Rev Pharmacoecon Outcomes R

Short description: 

This is a scoping review of the volume, economic analytic techniques and therapeutic areas encompassed by the pediatric economic evaluation studies published between 1980 and 2014.  The review uses records from the Paediatric Economic Database Evaluation (PEDE) database of pediatric economic evaluations to explore trends in analytical techniques, target age groups, interventions, outcomes and disease classifications. The paper describes the eligibility criteria for PEDE. PEDE is populated through searches of  MEDLINE, CINAHL, EMBASE, International Pharmaceutical Abstracts (IPA), EconLit, the Cochrane Library, NHS EED and ERIC, as well as searches of over 73 websites of HTA agencies and research groups. PEDE is updated annually.

Limitations stated by the author(s): 

Studies might have been missed for inclusion in PEDE if adolescents were included in adult populations.  Non-English language papers are not eligible unless they have an English abstract.

Limitations stated by the reviewer(s): 
No other limitations were detected by the reviewers.

Appraisal of: Thielen FW, Van Mastrigt G, Burgers LT, Bramer WM, Majoie H, Evers S, Kleijnen J. How to prepare a systematic review of economic evaluations for clinical practice guidelines: database selection and search strategy development (part 2/3). Exp

Short description: 

This article is the second in a three-part series on how to prepare a systematic review of economic evaluations. It provides a good overview of the literature on how to select relevant databases and develop a search strategy for retrieving economic evaluations. While the main target audience of the article is developers of clinical practice guidelines, the process described is helpful for systematic review researchers as well as those undertaking health technology assessments. The authors identify four steps in the search process: (1) selecting relevant databases (basic, specific and optional); (2) developing a comprehensive search strategy; (3) performing the searches (including documentation), and (4) selecting the relevant studies. The authors note the recent discontinuation of two health economics databases (HEED and NHS EED), which has resulted in an increased reliance on the use of search filters designed to capture economic evaluations. Also noted are unsettled issues, such as the lack of consensus on how many and which databases should be searched, as well a lack of uniform guidance on the methodology of developing a sound search strategy. Validated search filters and automated processes may help to overcome problems created by the lack of health economics-specific databases.

Limitations stated by the author(s): 

No limitations stated by the study authors.

Limitations stated by the reviewer(s): 
No additional limitations detected by the reviewers.
Study Type: 
Review

Appraisal of: Wood H, Arber M, Glanville JM. Systematic reviews of economic evaluations: how extensive are their searches? Int J Technol Assess Health Care. 2017 Mar 27:1-7.

Reviewer(s): 
Short description: 

This study evaluated the search methodology of recent systematic reviews of economic evaluations. A sample of 42 reviews identified through a MEDLINE search was analyzed. The analysis included: databases searched (general & specialist), health technology assessment sources searched, and supplementary search techniques used. The search approaches used in the systematic reviews were compared to two current recommendations: 1) the minimum search resources from the National Institute for Health and Care Excellence (NICE) economic search requirements for single technology appraisals; 2) the resources recommended in the costs and economic evaluation chapter of SuRe Info. A majority (55%) of the reviews did not meet either the NICE or SuRe Info recommendations. The data collection was hindered by lack of clarity and errors in search methodology reporting within the systematic reviews. It is likely that current recommendations on searching for economic evaluations will change due to the recent closure of two specialized databases (NHS EED & HEED). 

Limitations stated by the author(s): 

The sample of systematic reviews was pragmatic and limited to papers which were freely available or available through the authors' subscription access. Additionally, the sample was limited to English language systematic reviews only. Reviews were determined to be systematic reviews if the review authors identified them as such; however, it is possible that the review authors may have mislabeled narrative reviews as systematic.  Because of lack of clarity in search methodology reporting, reviews that vaguely mentioned searching "The Cochrane Library" were assumed to have searched all databases contained by this resource, which may have overestimated the use of the NHS EED and HTA databases.

Limitations stated by the reviewer(s): 
No additional limitations detected by the reviewers.
Study Type: 
Single study

Appraisal of: "De Cock E, Cosmatos I, Kirsch E. Use of databases for health resource utilization and cost analyses in EU-5: results from a focused literature review [abstract and poster]. Value Health 2016;19(3):A80-A81"

Short description: 
This conference abstract describes the use of databases to identify health resource utilization (HRU).  The authors searched PubMed/MEDLINE and Embase for secondary care, health resources/utilisation, databases, and EU-5 (Great-Britain (GB), France, Germany, Italy, and Spain) published between 2011 and 2015. 146 abstracts were initially retrieved and 48 were eligible with settings in GB, Italy, Germany, France, and Spain. The type of database differed by country. The most frequently listed resource items were inpatient stays, specialist visits and pharmacotherapy. The use of databases depends on each country's health system.
 
Limitations stated by the author(s): 

The abstract was probably too brief for the authors to be able to explore limitations.

Limitations stated by the reviewer(s): 
The available information is limited because this was presented at a conference. The focus was on the EU5 countries and limited to five years, but in the absence of other published studies on the identification of resource use data, the full report is awaited with interest.
Study Type: 
Single study
Related Chapters: 

Appraisal of: Paisley S. Identification of evidence for key parameters in decision-analytic models of cost effectiveness: a description of sources and a recommended minimum search requirement [epub]. PharmacoEconomics. 2016. DOI 10.1007/s40273-015-0372-x

Short description: 

This paper provides recommendations for the minimum level of searching which should be conducted to populate the various parameters  (e.g. treatment effects, costs, resource use) for a health economic decision model. The minimum searching level varies by parameter and the author suggests that the level should be adjusted depending on the importance of the parameter to the model. The paper describes the range of different resources which might need to be searched for each parameter and also indicates where country or region-specific resources might need to be identified. Specific guidance is given for each parameter and where more detailed guidance is available for search approaches for specific parameters, this guidance is signposted. This paper also makes brief recommendations on how to report searches.

Limitations stated by the author(s): 

The author highlights the developmental nature of the searches undertaken to inform decision model searches since modelling is an “iterative and dynamic process” and that this is difficult to capture in procedural guidance such as this.  The author acknowledges that the recommendations in this guidance have “not been tested empirically” and further research is required to support the recommendations.

Limitations stated by the reviewer(s): 
No additional limitations were detected by the reviewers.
Study Type: 
Single study

Appraisal of: Pitt C, Goodman C, Hanson K. Economic evaluation in global perspective: a bibliometric analysis of the recent literature. Health Econ. 2016 Feb;25 Suppl 1:9-28.

Reviewer(s): 
Short description: 

A bibliometric analysis of full economic evaluations of health interventions was conducted. A search of 14 databases identified 2844 economic evaluations published between 1 January 2012 and 3 May 2014. Analyses included: the sensitivity and specificity of searches in all databases, the distribution of the publications between countries, regions and health areas studied, as well as authors' country and institutional affiliations, journal type, language, and type of economic evaluation. The Scopus search identified the largest number of economic evaluations (85%), followed by NHS EED (80%), and MEDLINE (79%). EconLit captured only 1% of the publications identified. The majority of economic evaluations were published in biomedical journals (74%), addressed high-income countries (83%), and had at least one author based in a high-income country (91%). The health areas studied correlated more closely with the burden of diseases in high income countries compared to low-and middle-income countries. 

Limitations stated by the author(s): 

The NHS EED database is no longer being updated as of March 2015. Wiley HEED ceased to be available at the end of 2014. The authors assigned one point per institution per article in the analysis of institutional affiliation, which gives more weight to articles from multiple institutions. A single article could be classified as belonging to multiple health areas, which could be considered double-counting.

Limitations stated by the reviewer(s): 
The search strategies used to identify economic evaluations were unique strategies created for this study and were not established search filters, so they do not reflect current practice of information specialists. Because the search strategies were very specific, they may have missed relevant studies which were not explicit about being an economic evaluation. Additionally, variations in the search strategies between databases may have affected sensitivity and precision of retrieval. For example, in the cross-disciplinary databases strategies included a concept to limit retrieval to health topics. As such, the results presented might not fully represent the content of the databases. Because of these factors, it may be difficult to draw meaningful conclusions based on the data presented in this article.
Study Type: 
Single study
Related Chapters: 
Syndicate content