Evidence collections

Scope: use to index papers investigating the benefits and uses of collections of research evidence arranged by clinical themes or evidence types where the information presented is synthesized from a range of evidence resources using a specified methodology, e.g. Clinical Evidence

Appraisal of: "Booth A. Searching for qualitative research for inclusion in systematic reviews: a structured methodological review. Syst Rev, 5(1). doi:10.1186/s13643-016-0249-x"

Reviewer(s): 
Short description: 

This review provides a systematic overview of the available published evidence of searching methods to inform qualitative evidence synthesis (QES). The author sought to assess and identify:

1) the current state of knowledge in relation to searching for qualitative evidence

2) the robustness of the evidence base

3) research gaps and future priorities.

The studies were obtained from the Reference Manager database of the Cochrane Qualitative Methods Group’s study register, of which the author is responsible for updating and maintaining. Supplementary citation searches via Google Scholar was also carried out for 15 key papers. 113 studies were assessed for inclusion. Quality assessment of the included studies was not deemed feasible due to a large proportion of the included studies providing only narrative findings, the lack of a common appraisal instrument and the high levels of heterogeneity across the remaining studies.

The evidence underpinning systematic approaches to searching for qualitative evidence is classified and summarized within one or more of eight headings/ “7 S structure/ framework”: overviews and methodological guidelines, sampling, sources, structured questions, search procedures, search strategies and filters, supplementary strategies and standards. The author summarizes the available evidence and key issues within each section and makes recommendations for further empirical research. Table 7 breaks down the key starting principles in reference to the “7S structure” of searching to inform qualitative evidence synthesis to inform future guidance and Table 8 provides an overview of research priorities.

The review concludes that there is a lack of empirical data to inform information retrieval for QES and that the strength of the evidence is weak and largely based on personal/ professional experience and case studies. Advances have been made in reporting QES, however, validated standards are lacking. 

Limitations stated by the author(s): 

The author states that in order for studies to be included, the references needed to include terms specifically related to searching or retrieval in their titles or abstracts, cite a number of key texts, or be referred to from previously identified items. The full-text of all papers reporting QES were not examined. There is a possibility that potentially relevant reviews reporting emerging information retrieval methods that were not reported in the title or abstract were missed. However, these risks are offset by the sensitive search approach and the currency and comprehensiveness of the Cochrane Qualitative Methods Group study register. The author also notes that some papers were excluded as they did not distinguish between qualitative and quantitative approaches, which could potentially be useful for mixed methods reviews.

Limitations stated by the reviewer(s): 
1. This was a single reviewer methodology review so judgements on eligibility and interpretations of potential significance and contribution of individual studies may not be consistent and/or reproducible. [Study Selection Bias] 2. Although as a methodology review there is no formal requirement to follow PRISMA reporting standards this review may have benefited from more complete and transparent reporting. [Incomplete Reporting Bias] 3. As the reviewer was author on a high proportion of included studies this may have consciously impacted on study identification and subconsciously on study selection and interpretation. [Citation Bias; Observer Bias]
Study Type: 
Single study

Appraisal of: Greenhalgh T, Peacock R. Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ 2005; 331(7524): 1064-1065.

Short description: 

To describe where papers come from in a systematic review of complex evidence. Only 30% of sources were obtained from the protocol defined at the outset of the study (that is, from the database and hand searches). Fifty one per cent were identified by “snowballing” (such as scanning reference lists (44%) and citation tracking (7%)), and 24% by personal knowledge or personal contacts.

Limitations stated by the author(s): 

No limitations were stated by the authors.

Limitations stated by the reviewer(s): 
The article assesses the total contribution of the different sources but it is not possible to identify the additional references which could only be found by the additional search techniques. Besides that it is not mentioned if perhaps the original Boolean search was inadequate as the searches and the results are not discussed in detail. The generalizability of the results is also questionable because the authors results are based only on one review (topic “diffusion of service-level innovations in healthcare organizations”).
Study Type: 
Single study

Appraisal of: Glanville J, Paisley S. Identifying economic evaluations for health technology assessment. Int J Technol Assess Health Care 2010;26(4):436-440.

Reviewer(s): 
Short description: 
This study surveys the current evidence base for search methods and resources for retrieving economic evaluations in the context of health technology assessment (HTA) work. To find economic evaluations, specific economic evaluation databases and larger biomedical databases must be searched to obtain the most comprehensive sets of results. In addition to a list of key databases, strategies for searching both types of databases are presented, along with other general database search tips. Evidence on the performance of several economic evaluation search filters is summarized, with a call for further evidence-generating research in this area.
Limitations stated by the author(s): 
The authors state that there is a lack of evidence supporting best practice in retrieving economic evaluations, such as efficient grey literature searching, the development of search filters, and “comparative yield” of different search approaches and resources.
Limitations stated by the reviewer(s): 
Non-database sources appear to have been surveyed in the study, but the authors do not elaborate on any identified.
Study Type: 
Review
Related Chapters: 

Appraisal of: Glanville J, Paisley S. Chapter 7: Searching for evidence for cost-effectiveness decisions. In: Shemilt I, Mugford M, Marsh K, Donaldson C. (editors). Evidence-based decisions and economics: health care, social welfare, education and crimin

Reviewer(s): 
Short description: 
This textbook chapter summarizes the available published evidence on the approaches and methods for literature searching in support of both economic evaluations and decision models in health care. For economic evaluations, a range of database resources and search filter designs are presented. For decision models, the inappropriateness of the traditional PICO search approach is discussed. The authors provide an outline of the unique types of information needs and resources required for these models. Systematic approaches and transparency are emphasized, and an understanding of the development and purpose of both economic evaluations and decision models is encouraged.
Limitations stated by the author(s): 
The chapter authors focus their analysis to health care related economic evaluations and decision models only. Searches on other peripheral topics in the health economics area, such as burden of illness studies, health resources use, etc. are deferred.
Limitations stated by the reviewer(s): 
No additional limitations detected by the reviewer.
Study Type: 
Review
Related Chapters: 
Syndicate content