Databases

Scope: provide the database name(s); add to records where a database rather than a website or interface is discussed

Appraisal of: Nixon J, Duffy S, Armstrong N, Craig D, Glanville J, Christie J, Drummond M, Kleijnen J. The usefulness of the NHS Economic Evaluation Database to researchers undertaking technology assessment reviews. Int J Technol Assess Health Care 2004;

Reviewer(s): 
Short description: 
This article reports the findings of a qualitative survey on the usefulness of NHS EED to lead authors of NICE Technology Assessment Reviews (TARs). The survey was conducted by mail and included questions on usefulness in identifying economic studies, extracting data, completing quality assessments, and informing decisions to commission modeling studies. NHS EED was used in 90% of all identified TARs and found to be generally useful, especially for retrieving economic evidence (62% of respondents), followed by literature quality assessment. However, a high proportion didn’t find NHS EED relevant to quality assessment and data extraction as the TAR protocol is to use the primary study. The time lag between introduction of a technology, the publication of an economic evaluation, and the article abstraction for NHS EED is also discussed.
Limitations stated by the author(s): 
Low survey response rate (63%). Respondents may have been motivated to complete the survey by a lack of economic evaluations in their topic areas.
Limitations stated by the reviewer(s): 
No additional limitations detected by the reviewer.
Study Type: 
Single study
Related Chapters: 

Appraisal of: Golder S, Glanville J, Ginnelly L. Populating decision-analytic models: the feasibility and efficiency of database searching for individual parameters. Int J Technol Assess Health Care 2005 Summer;21(3):305-311.

Reviewer(s): 
Short description: 
The authors investigated the efficacy of searching selected databases to retrieve specific information to populate a decision-analytic (economic) model. Altogether 42 specific questions were developed from a single case study and then grouped into 18 search strategies. Information required to populate the model for the case study was divided into five categories: baseline event rates, health-related quality of life, resource use and cost units, relative treatment effects and antibiotic resistance. Databases searched included MEDLINE, Embase, the Incidence and Prevalence Database (IPD), Cochrane Database of Systematic Reviews (CDSR), the Database of Abstracts of Reviews of Effects (DARE), the Health Technology Assessment (HTA) database, Cochrane Central, NHS Economic Evaluation Database (NHS EED) and the Health Economic Evaluations Database (HEED). Results were presented describing the most useful databases for each of the five question types. Issues and strategies in developing searches for each question type were also discussed. Searching multiple databases using focused search strategies for each question resulted in the identification of important data for inclusion in the model. It was noted that while searches of this nature may not be highly sensitive, the search process nevertheless can still be systematic, transparent and well-documented.
Limitations stated by the author(s): 
The paper describes only one case study which retrieved a limited number of relevant citations, making it difficult to generalize to other decision-analytic models. The information required to populate a different decision-analytic model may require an adjustment to resources and strategies used. Project time and resources will dictate the acceptable level of focus in the search strategies as well as the number of source searched.
Limitations stated by the reviewer(s): 
National or regional sources of information may be required to find some relevant data to populate models. Grey literature searching is not discussed.
Study Type: 
Single study
Related Chapters: 

Appraisal of: Glanville J, Paisley S. Chapter 7: Searching for evidence for cost-effectiveness decisions. In: Shemilt I, Mugford M, Marsh K, Donaldson C. (editors). Evidence-based decisions and economics: health care, social welfare, education and crimin

Reviewer(s): 
Short description: 
This textbook chapter summarizes the available published evidence on the approaches and methods for literature searching in support of both economic evaluations and decision models in health care. For economic evaluations, a range of database resources and search filter designs are presented. For decision models, the inappropriateness of the traditional PICO search approach is discussed. The authors provide an outline of the unique types of information needs and resources required for these models. Systematic approaches and transparency are emphasized, and an understanding of the development and purpose of both economic evaluations and decision models is encouraged.
Limitations stated by the author(s): 
The chapter authors focus their analysis to health care related economic evaluations and decision models only. Searches on other peripheral topics in the health economics area, such as burden of illness studies, health resources use, etc. are deferred.
Limitations stated by the reviewer(s): 
No additional limitations detected by the reviewer.
Study Type: 
Review
Related Chapters: 

Appraisal of: Coyle KB, Trochlil K, Iversen P. MEDLINE and EMBASE for health economic literature reviews [abstract]. Value Health 2012;15(4):A162.

Reviewer(s): 
Short description: 
The authors compared search results in MEDLINE and Embase for the indication chronic pulmonary obstructive disease (COPD) with regard to cost of illness (COI), burden of illness (BOI) and cost benefit analysis (CBA). MEDLINE retrieved a higher number of references for CBA, while Embase retrieved higher results in the areas of COI and BOI. Conference abstracts accounted for the higher Embase totals. The inclusion of conference abstracts in Embase support the practice of conducting economic literature reviews in both MEDLINE and Embase.
Limitations stated by the author(s): 
None stated - this is an abstract.
Limitations stated by the reviewer(s): 
Since the results were published only in abstract form, very limited information was presented on methods, results and conclusions. Specific search strategies were not provided, nor was it stated if the citations retrieved were evaluated for relevance. Only one subject area (COPD) was searched for citations published during 2010-11, resulting in low retrieval results. Differences in retrieval may therefore not have statistical significance or be generalizable to other topic areas.
Study Type: 
Single study
Related Chapters: 

Appraisal of: Alton V, Eckerlund I, Norlund A. Health economic evaluations: how to find them. Int J Technol Assess Health Care 2006; 22(4):512-517.

Reviewer(s): 
Short description: 
This article investigates databases and search terms to retrieve the most relevant published health economic evaluation studies. The authors searched six bibliographic databases (NHS EED via Cochrane Library and via CRD; PubMed; Embase, EconLit, SciSearch via DialogClassic; and HEED on CD-ROM) using various search strategies. Searching NHS EED plus PubMed was determined to be the best approach. PubMed did not consistently index cost-effectiveness studies, so a combination of the MeSH “Costs and Cost Analysis”, the MeSH subheading “Economics”, and text words were used in the filter.
Limitations stated by the author(s): 
This is a case study where approaches were tested for only one topic area: diagnosing GERD. Economic evaluations retrieved that were irrelevant to this topic were excluded. Studies weren’t assessed for quality and internal validity as it was beyond the scope.
Limitations stated by the reviewer(s): 
The detailed search strategies and date limits are not published in the article; readers are referred to the SBU website. MeSH terms and text words are suggested, but not rigorously or individually tested.
Study Type: 
Single study
Related Chapters: 
Supplemental publications to the study: 

Appraisal of: Royle P, Waugh N. Literature searching for clinical and cost-effectiveness studies used inhealth technology assessment reports carried out for the National Institute for Clinical Excellence appraisal system. Health Technol Assess 2003;7(34).

Short description: 
This study assessed a sample of 20 NICE technology assessment reports (TARs) to assess optimum literature retrieval strategies for identifying clinical and cost-effectiveness studies using relative recall methods. All sources searched for the TARs were identified and 130 included cost-effectiveness studies were identified. The presence of the included studies within sources was recorded. 73.1% of studies were published and 1.5% were abstracts. 86.6% of those published/abstract studies were indexed in MEDLINE, 86.6% in Embase and 40.2% in NHS EED. Searching SCI and ASCO Online in addition increased retrieval of published studies to 97.9%. Searching non-database sources an identify further studies: 25% of the included studies were ‘unpublished’ or grey literature.
Limitations stated by the author(s): 
The number of TARs assessed was small and may be biased by the large number of cancer topics. Most of the key measurement decisions were made by only one reviewer. The descriptions of study designs and search accuracy were accepted uncritically. The effect of the harder to find studies on the effect size was not assessed. The search strategies were not assessed for relevance and fitness for purpose: their quality is unknown. The TARs may not be generalisable to the future TAR programme in terms of subject focus.
Limitations stated by the reviewer(s): 
The study was conducted on reviews published in 2000-2002. The relative recall approach adopted gives an assessment of the relative yield of databases and their overlap with a perfect search strategy. The study did not rerun TAR search strategies to investigate how many of the studies which are available to be found, would have been found by the strategies used.
Study Type: 
Single study
Related Chapters: 

Appraisal of: McKinlay RJ, Wilczynski NL, Haynes RB, Hedges Team. Optimal search strategies for detecting cost and economic studies in EMBASE. BMC Health Serv Res 2006;6:67.

Short description: 
This study reported the development of a search strategy for cost and economics studies in Embase. The gold standard was created by handsearching 55 journals for the year 2000. The authors provide strategies to find studies about costs and about economic studies broadly defined. A cost strategy with a sensitivity of 100% and precision of 8.6% is presented along with an economic studies strategy of 100% with 1.4 % precision. Best compromise strategies are offered for costs (sensitivity 98.4% and precision 18.2%) and economics (sensitivity 96.8% and precision 4.3%)
Limitations stated by the author(s): 
Precision will be lower when searching the entire Embase database. Multivariate statistical techniques might yield better performing strategies but authors did test a logistic regression approach.
Limitations stated by the reviewer(s): 
55 journals were chosen for frequency of yield and these may be different to the wider range of journals which publish cost and economics studies. The handsearched records are for one year only. High impact factor journal were chosen and these may be different to other journals perhaps in terms of encouraging better study reporting. The authors have a broad definition of economics which means their strategy may not be optimized to find economic evaluations.
Study Type: 
Single study
Related Chapters: 

Appraisal of: Glanville J, Kaunelis D, Mensinkai S. How well do search filters perform in identifying economic evaluations in MEDLINE and EMBASE. Int J Technol Assess Health Care 2009;25(4):522-529.

Short description: 
This research was undertaken to develop search filters to identify economic evaluations in MEDLINE and Embase. The performance of these filters and published filters were tested in terms of sensitivity and precision using a gold standard set of known economic evaluations. Thirteen filters for MEDLINE were tested and three yielded a sensitivity of 99%. The best precision for one of these filters was 4%. The best compromise filter provided 84% sensitivity for 13% precision. Four filters provided greater than 99% sensitivity in Embase with precision ranging from 1.5- 2.9%. The best compromise filter provided 23% precision and 63% sensitivity.
Limitations stated by the author(s): 
The study relies on a gold standard obtained from NHS EED and therefore the NHS EED filter may overperform when tested against it and also how far NHS EED represents typical economic evaluations needs to be judged. It was not possible to assess whether the search filters identified economic evaluations not captured by NHS EED. The performance of filters for finding MEDLINE in process or PubMed records was not tested. Excluding animal studies improved precision in MEDLINE but not in Embase.
Limitations stated by the reviewer(s): 
No additional limitations detected by the reviewer.
Study Type: 
Single study
Related Chapters: 

Appraisal of: Golder S, Loke YK. Failure or success of electronic search strategies to identify adverse effects data. J Med Libr Assoc 2012;100(2):130-134.

Short description: 
The aim of this study was to assess the prevalence of adverse effects terms in the title, abstract, and / or indexing of adverse effects papers published from 2001 onwards and the implications of this on search strategy approaches. A cohort of 242 papers, indexed in MEDLINE, Embase and / or Science Citation Index (SCI), was identified to investigate this and compare the results with a similar study published in 2001 by Derry, Loke and Aronson. Of the 242 papers identified, 231 were indexed in MEDLINE, 222 were indexed in Embase and 238 were indexed in SCI. Searching with generic or specific named adverse effects terms in the title, abstract, or indexing identify 89% of the references indexed in Embase, 80% of the references indexed in Medline and 70% of the references indexed in SCI. Generic adverse effects terms in the title and abstract in any of the 3 databases, generic and specific indexing terms or adverse effects subheadings in Embase, and subheadings for adverse effects in Medline retrieved the highest proportion of references. Compared to previous research findings, adverse effects terms seemed to be increasingly prevalent in the title, abstract and indexing of adverse effects papers in both MEDLINE and Embase.
Limitations stated by the author(s): 
No limitations stated by the study authors
Limitations stated by the reviewer(s): 
No limitations detected by the reviewer
Study Type: 
Single study
Related Chapters: 

Appraisal of: Golder S, Loke YK. Sensitivity and precision of adverse effects search filters in MEDLINE and EMBASE: a case study of fractures with thiazolidinediones. Health Info Libr J 2012;29(1):28-38.

Short description: 
A systematic review of fracture-related adverse effects associated with the use of thiazolidinediones was used as a case study to calculate the sensitivity, precision and Number Needed to Read (NNR) of published adverse effects search filters in MEDLINE and Embase. 12 MEDLINE filters and 3 Embase filters were tested. The results showed that 4 search filters in MEDLINE achieved high levels of sensitivity (95 or 100%) with improved levels of precision compared with searches without any adverse effects filters. The highest level of precision in MEDLINE (55%) was achieved with search filters that relied only on subject headings (MeSH). No search filter in Embase achieved sensitivity higher than 83% and precision remained low using any of the filters (all under 5%).
Limitations stated by the author(s): 
The main limitation to this study is that because only one systematic review was used as a case study, the generalisability of the study results is limited. In addition, this case study is of a particular named adverse effect (fractures), while a case study of a safety profile systematic review, in which all adverse effects are searched for, might have given different results. Another limitation is the adaptations made to some of the tested search filters. These filters were originally created for use in searches where the adverse effects are not known in advance of searching, while in this case study, these filters were used in addition to search terms for a named adverse effect (fracture terms).
Limitations stated by the reviewer(s): 
In addition to the limitation noted above by the authors, it should be noted that the number of included studies on which the search filters were tested was relatively small (19 records in MEDLINE and 24 records in Embase). It should also be noted that the filters were tested as published. Testing of individual terms within the filters might have resulted in higher precision and better trade-off between sensitivity and precision than the full filters.
Study Type: 
Single study
Related Chapters: 

Comments from the authors:

Syndicate content