Costs and economic evaluation

We are grateful for the assistance of Eleanor Kotas on the chapter in 2019-2020 and Kelly Farrah in 2020.


This domain focuses on the importance of obtaining information about costs and outcomes as well as efficacy and effectiveness when evaluating new technologies. Economic evaluation is an important part of health technology assessment because it assists with priority-setting between different health technologies. An economic evaluation identifies, measures, values and compares the costs and outcomes of a technology with its relevant comparator.

This domain overlaps with the effectiveness domain and the organizational domain (1).

Guidance on conducting searching as part of systematic reviews of economic evaluations and utilities have recently been published (2,3).

Sources to search

There are some databases which identify and collect economic evaluations and health economics studies (4,5,6,7,8,9) to promote efficient retrieval. These databases are built largely from MEDLINE and Embase, but offer a variety of value added information such as critical appraisals, results, categorisations and indexing. These databases can save time in identifying economic evaluations, but may not be comprehensive because of publication lags or geographical focus (e.g. the Cost-Effectiveness Analysis (CEA) registry). NHS EED ceased updating at the end of 2014 and is available only as a closed database. HEED is no longer available. This means that sensitive searches should also include searches of general medical databases such as MEDLINE and Embase (4,5,8,9,10,11,12). Searching Science Citation Index and conference abstracts (via websites as well as Embase) may also increase retrieval (10,13). Pitt et al. conducted a bibliometric analysis of full economic evaluations of health interventions published in 2012-14, comparing, among other things, the sensitivity and specificity of searches in 14 databases (14). This study confirms that Econlit is not a high yield resource for economic evaluations and suggests that Scopus may be a useful resource to search, which may merit investigation.

Searching non-database sources is likely to identify further studies outside of commercial journal publications (10).

The majority of recent reviews of economic evaluations have not followed published searching approaches in detail and are also currently poorly reported (15). Reviews should report the searches explicitly and search a range of resources (2,9,15). The following information sources should be considered when searching for economic evaluations and utility studies:

Identifying information to populate economic models may involve searching sources ranging from statistical resources to bibliographic databases (4,5,21,22,23,24,25). Guidance on suggested minimum searching levels for model parameters is available, although the author notes that much of the guidance has not been empirically tested (25). Additional suggestions for identifying utility studies, include standard approaches such as checking the reference lists of eligible studies, consulting experts, carrying out citation searches and named author searching (3).  One study has examined the use of routine data (typically obtained for health insurance funds or other reimbursement data sources rather than bibliographic databases) in economic evaluations and highlighted that these data may increasingly need to be included in economic evaluations (26).

Designing search strategies

Principles of systematic review methodology should be followed for the design of search strategies to identify economic evaluations. The development of sensitive subject searches within the specific economic evaluation databases is recommended to capture the population and the intervention of interest (4,5,27). An overview of methods for systematic reviews of health economic interventions suggests that a systematic search should use relevant elements of PICO combined with an economic search filter (18). Shemilt is more cautious still, suggesting that only intervention search terms may be required and focus can be achieved by adding the population concept (16). However, there is no requirement to add an economic evaluation search filter to searches within economic evaluation databases because they are pre-filtered (3,7). Search filters for economic studies can be considered (in combination with concepts capturing the population and/or intervention) in general bibliographic databases such as MEDLINE or Embase (18). Published search filters, which can be identified from the InterTASC Information Specialists' Sub-Group (ISSG) Search Filter Resource, tend to have high sensitivity but poor precision (28,29,30). A filter validation study conducted in 2018 highlights Embase filters with between 89.9% (2.9% precision) and 70.2% sensitivity (14.1% precision) (31). CADTH offers a more precision maximizing search filter for rapid reviews (32). Search strategies to identify cost-effectiveness information may need to be adapted from those developed for searching for effectiveness studies (33). Searching for particular economic methods may require the use of several techniques (34).

Searches to inform specific parameters of decision models may not be required to be as extensive and systematic as those to identify economic evaluations, as decision models are developed in an organic way, some parameters do not require the identification of comprehensive evidence and also it may not be feasible to conduct extensive searches for all parameters of a model (4,22,25).

Health state utility values (HSUVs) are important parameters in decision models and searching for them requires specific techniques (3,21,35) and the careful use of search filters can also be considered (3,36). There are few subject headings dedicated to utilities within MeSH and EMTREE, and although general subject headings such as ‘Quality of life’ will yield relevant studies, they are likely to demonstrate poor precision (3). Free text terms should be included in searches and three types may be helpful to include: general terms (such as QALY), instrument specific terms (such as EQ-5D) and terms describing methods of utility elicitation such as standard gamble (3). Search filters make use of a selection of these terms which have been shown to perform well in practice (3,34,36). Arber et al. have published three validated filters for retrieving HSUVs (sensitivity maximizing; a balance of sensitivity and precision; and precision maximizing) (36). One study recommends the use of iterative searching for utilities, following an initial scoping search, and lists factors to consider when defining the search criteria (35).

Searching for cost of illness/burden of illness can make use of population search terms (perhaps taken from an accompanying effects review) (16).


Reference list