Journal articles

Scope: use to index descriptions of this publication type, collections of journal articles, how to identify journal articles or investigations of the contribution of journal articles to HTA

Appraisal of: Wilczynski NL, Morgan D, Haynes RB; Hedges Team. An overview of the design and methods for retrieving high-quality studies for clinical care. BMC Med Inform Decis Mak. 2005 Jun 21;5:20.

Reviewer(s): 
Short description: 

The authors describe the methods they use to develop optimal search filters (hedges) to identify a range of study types: causation, prognosis, diagnosis, treatment, economics, clinical prediction guides, reviews, costs, and qualitative:  the filters were designed for MEDLINE and Embase. The authors built a large gold standard (reference set) by hand searching 170 journals for one year: 2000.  Relevant records were defined and were selected to represent best research methods. The gold standard records were then downloaded from  MEDLINE, Embase, CINAHL, and PsycINFO with the subject indexing assigned by each database.  Candidate search terms were identified from the gold standard records and consulting experts. The sensitivity, specificity, precision, and accuracy of unique search terms and combinations of search terms were calculated. Once the performance parameters of individual search terms were computed, the authors  selected individual terms for the construction of search strategies by choosing search terms with specific levels of sensitivity and specificity (which varied by database). The authors also used logistic regression to explore ways to improve filter performance. Strategies were developed in a random selection of 60% of the gold standard and validated in the remaining 40%. No statistical differences in performance were found between the two strategy development methods or between the test and validation results, so the majority of filter development used the Boolean approach and search strategies were developed using all records in the database.

The gold standard database numbered 60,330 records, each with up to 11 data fields. Filters were developed for studies of causation, prognosis, diagnosis, treatment, economics, clinical prediction guides, reviews, costs, and those of a qualitative nature.

Limitations stated by the author(s): 

The authors do not state any limitations to their methods.

Limitations stated by the reviewer(s): 
The journals contributing to the gold standard were largely general (internal medicine, general medical practice, mental health and general nursing practice) rather than specialty focused. There may be issues about generalizability to the latter journals. Records were selected for the gold standard according to the authors’ definition of the specific topics and best study designs for those topics. These definitions need to be compared to the definitions of any user. The authors decided, following testing, to use all of the database records to both develop and test the filters. Ideally validation data on a different reference standard should be provided.
Study Type: 
Single study

Appraisal of: Nixon J, Duffy S, Armstrong N, Craig D, Glanville J, Christie J, Drummond M, Kleijnen J. The usefulness of the NHS Economic Evaluation Database to researchers undertaking technology assessment reviews. Int J Technol Assess Health Care 2004;

Reviewer(s): 
Short description: 
This article reports the findings of a qualitative survey on the usefulness of NHS EED to lead authors of NICE Technology Assessment Reviews (TARs). The survey was conducted by mail and included questions on usefulness in identifying economic studies, extracting data, completing quality assessments, and informing decisions to commission modeling studies. NHS EED was used in 90% of all identified TARs and found to be generally useful, especially for retrieving economic evidence (62% of respondents), followed by literature quality assessment. However, a high proportion didn’t find NHS EED relevant to quality assessment and data extraction as the TAR protocol is to use the primary study. The time lag between introduction of a technology, the publication of an economic evaluation, and the article abstraction for NHS EED is also discussed.
Limitations stated by the author(s): 
Low survey response rate (63%). Respondents may have been motivated to complete the survey by a lack of economic evaluations in their topic areas.
Limitations stated by the reviewer(s): 
No additional limitations detected by the reviewer.
Study Type: 
Single study
Related Chapters: 

Appraisal of: Golder S, Glanville J, Ginnelly L. Populating decision-analytic models: the feasibility and efficiency of database searching for individual parameters. Int J Technol Assess Health Care 2005 Summer;21(3):305-311.

Reviewer(s): 
Short description: 
The authors investigated the efficacy of searching selected databases to retrieve specific information to populate a decision-analytic (economic) model. Altogether 42 specific questions were developed from a single case study and then grouped into 18 search strategies. Information required to populate the model for the case study was divided into five categories: baseline event rates, health-related quality of life, resource use and cost units, relative treatment effects and antibiotic resistance. Databases searched included MEDLINE, Embase, the Incidence and Prevalence Database (IPD), Cochrane Database of Systematic Reviews (CDSR), the Database of Abstracts of Reviews of Effects (DARE), the Health Technology Assessment (HTA) database, Cochrane Central, NHS Economic Evaluation Database (NHS EED) and the Health Economic Evaluations Database (HEED). Results were presented describing the most useful databases for each of the five question types. Issues and strategies in developing searches for each question type were also discussed. Searching multiple databases using focused search strategies for each question resulted in the identification of important data for inclusion in the model. It was noted that while searches of this nature may not be highly sensitive, the search process nevertheless can still be systematic, transparent and well-documented.
Limitations stated by the author(s): 
The paper describes only one case study which retrieved a limited number of relevant citations, making it difficult to generalize to other decision-analytic models. The information required to populate a different decision-analytic model may require an adjustment to resources and strategies used. Project time and resources will dictate the acceptable level of focus in the search strategies as well as the number of source searched.
Limitations stated by the reviewer(s): 
National or regional sources of information may be required to find some relevant data to populate models. Grey literature searching is not discussed.
Study Type: 
Single study
Related Chapters: 

Appraisal of: Glanville J, Paisley S. Identifying economic evaluations for health technology assessment. Int J Technol Assess Health Care 2010;26(4):436-440.

Reviewer(s): 
Short description: 
This study surveys the current evidence base for search methods and resources for retrieving economic evaluations in the context of health technology assessment (HTA) work. To find economic evaluations, specific economic evaluation databases and larger biomedical databases must be searched to obtain the most comprehensive sets of results. In addition to a list of key databases, strategies for searching both types of databases are presented, along with other general database search tips. Evidence on the performance of several economic evaluation search filters is summarized, with a call for further evidence-generating research in this area.
Limitations stated by the author(s): 
The authors state that there is a lack of evidence supporting best practice in retrieving economic evaluations, such as efficient grey literature searching, the development of search filters, and “comparative yield” of different search approaches and resources.
Limitations stated by the reviewer(s): 
Non-database sources appear to have been surveyed in the study, but the authors do not elaborate on any identified.
Study Type: 
Review
Related Chapters: 

Appraisal of: Glanville J, Paisley S. Chapter 7: Searching for evidence for cost-effectiveness decisions. In: Shemilt I, Mugford M, Marsh K, Donaldson C. (editors). Evidence-based decisions and economics: health care, social welfare, education and crimin

Reviewer(s): 
Short description: 
This textbook chapter summarizes the available published evidence on the approaches and methods for literature searching in support of both economic evaluations and decision models in health care. For economic evaluations, a range of database resources and search filter designs are presented. For decision models, the inappropriateness of the traditional PICO search approach is discussed. The authors provide an outline of the unique types of information needs and resources required for these models. Systematic approaches and transparency are emphasized, and an understanding of the development and purpose of both economic evaluations and decision models is encouraged.
Limitations stated by the author(s): 
The chapter authors focus their analysis to health care related economic evaluations and decision models only. Searches on other peripheral topics in the health economics area, such as burden of illness studies, health resources use, etc. are deferred.
Limitations stated by the reviewer(s): 
No additional limitations detected by the reviewer.
Study Type: 
Review
Related Chapters: 

Appraisal of: Coyle KB, Trochlil K, Iversen P. MEDLINE and EMBASE for health economic literature reviews [abstract]. Value Health 2012;15(4):A162.

Reviewer(s): 
Short description: 
The authors compared search results in MEDLINE and Embase for the indication chronic pulmonary obstructive disease (COPD) with regard to cost of illness (COI), burden of illness (BOI) and cost benefit analysis (CBA). MEDLINE retrieved a higher number of references for CBA, while Embase retrieved higher results in the areas of COI and BOI. Conference abstracts accounted for the higher Embase totals. The inclusion of conference abstracts in Embase support the practice of conducting economic literature reviews in both MEDLINE and Embase.
Limitations stated by the author(s): 
None stated - this is an abstract.
Limitations stated by the reviewer(s): 
Since the results were published only in abstract form, very limited information was presented on methods, results and conclusions. Specific search strategies were not provided, nor was it stated if the citations retrieved were evaluated for relevance. Only one subject area (COPD) was searched for citations published during 2010-11, resulting in low retrieval results. Differences in retrieval may therefore not have statistical significance or be generalizable to other topic areas.
Study Type: 
Single study
Related Chapters: 

Appraisal of: Alton V, Eckerlund I, Norlund A. Health economic evaluations: how to find them. Int J Technol Assess Health Care 2006; 22(4):512-517.

Reviewer(s): 
Short description: 
This article investigates databases and search terms to retrieve the most relevant published health economic evaluation studies. The authors searched six bibliographic databases (NHS EED via Cochrane Library and via CRD; PubMed; Embase, EconLit, SciSearch via DialogClassic; and HEED on CD-ROM) using various search strategies. Searching NHS EED plus PubMed was determined to be the best approach. PubMed did not consistently index cost-effectiveness studies, so a combination of the MeSH “Costs and Cost Analysis”, the MeSH subheading “Economics”, and text words were used in the filter.
Limitations stated by the author(s): 
This is a case study where approaches were tested for only one topic area: diagnosing GERD. Economic evaluations retrieved that were irrelevant to this topic were excluded. Studies weren’t assessed for quality and internal validity as it was beyond the scope.
Limitations stated by the reviewer(s): 
The detailed search strategies and date limits are not published in the article; readers are referred to the SBU website. MeSH terms and text words are suggested, but not rigorously or individually tested.
Study Type: 
Single study
Related Chapters: 
Supplemental publications to the study: 
Syndicate content