Qualitative research

Last revised: 
2018-10-28

Introduction 

Qualitative research can make an important contribution to health technology assessment (1). This can involve the identification and review of qualitative evidence (qualitative evidence synthesis) designed to explain differences in intervention effect, to explore aspects of implementation or to capture evidence of patient experience or preference (2-4). Whilst the integration of qualitative evidence in health technology assessment is not common, its value is increasingly recognised as useful (2, 5).

The process of identifying qualitative research is less clear than for identifying studies reporting randomised controlled trials. Poor reporting of qualitative research in studies (3, 6, 7), limited indexing of studies (3, 8, 9), apparent confusion in the reporting of methods of data collection (interviews, focus groups) and synthesis (e.g. thematic synthesis, meta-ethnography) in studies (10, 11), and a need to search beyond primary biomedical databases (12, 13), are some of the reasons that qualitative research is more challenging to identify (2). Moreover, methodological guidance on ‘how to’ literature search for qualitative research is limited where it does exist (c.f. (3, 14)) (2).

Research supporting the process of searching for qualitative research was reviewed and summarised in a methodological review by Booth (2016) (2). This review forms the basis for the original version of this chapter.

At the outset of this chapter, we thoroughly recommend contacting an information professional or researcher with experience in literature searching for qualitative research.   

Sources to search

Bibliographic

A consensus has not been reached on the number of databases, or which databases, to be searched when conducting a literature search for qualitative research. Wright et al. (2014), in a retrospective case-study to consider the contribution of CINAHL when identifying qualitative research, indicated review teams searched between 3 and 20 databases to identify qualitative research (15).

The CRD handbook (3) currently recommends that searches should include the following databases for reviews and primary studies:

  • MEDLINE
  • Embase
  • CINAHL - Cumulative Index to Nursing and Allied Health
  • PsycINFO - psychological literature database

MEDLINE and CINAHL are the most frequently searched sources to identify qualitative research (16, 17) and their value has been demonstrated empirically (18). Dejean et al (2016) also indicate that the Social Science Citation index (SSCI, Clarivate Analytics) was a valuable resource for study identification (19). Reviewers are, however, strongly encouraged to identify specialist databases that relate to their review topic, together with databases that host particular types of publications or content e.g. dissertation abstracts, book chapters, theses and unpublished reports. Searching MEDLINE alone is considered inadequate. Where resources are limited the searcher should aim to access as diverse a sample of relevant databases as possible, for example capturing different disciplinary or professional perspectives and different literature types, as this will best further the objectives of the subsequent synthesis.

Supplementary search methods

Given the issues with identifying qualitative research (summarised above), supplementary search methods are considered particularly useful for identifying qualitative research (20). Supplementary search methods such as internet searching, personal contact with topic experts, handsearching and reference checking, may prove valuable since they may identify content not indexed in bibliographic databases and they do not necessarily rely on identifying precise search terms to identify studies and data (21). Campbell et al. recommend a multipronged approach of supplementary searching (11). Cooper et al (2017) have published a worked example of such an approach to literature searching for a qualitative evidence synthesis within a Cochrane public health systematic review (22).

Supplementary search methods have been summarised by Cooper et al (2017) (21).

Designing search strategies

Differences between conducting quantitative and qualitative research are mirrored in differences when designing literature searches for quantitative and qualitative literature (23). Whilst quantitative research and searches take a linear and structured route, qualitative research searches often use emerging results to inform subsequent strategies in a more iterative manner (23). As a consequence, qualitative searches may take a long time to plan, be more labour intensive to undertake and more challenging to replicate (2).

Search strategy design

Designing the search strategy may necessitate a different question formulation and search strategy structure from the Population, Intervention, Comparator, and Outcome (PICO) structure used by systematic reviews of intervention effectiveness (24). Booth summarises alternative structures to PICO below (see table 1).

 

Table 1. Alternative Notations for qualitative question formulation

Notation

Components

Source

Most Accessible variants

3WH

What (topical), Who (population), When (temporal), How (methodological)

(25)

PICo

Population, phenomenon of Interest, Context

(26)

Most Common variants

PICOC

Patient/Population, Intervention, Comparison, Outcomes, Context

(27)

SPICE

Setting, Perspective, Intervention/phenomenon of Interest, Comparison, Evaluation

(28)

Intervention based variants

(NB. A qualitative question may not always include a Comparison).

PICO

Patient/Population, Intervention, Comparison, Outcomes

(29)

PEICO(S)

Person, Environment, Intervention, Comparison, Outcomes, (Stakeholders)

(30)

PICOS

Patient/Population, Intervention, Comparison, Outcomes, Study type

(31)

Special Purpose variants

BeHEMoTh

Behaviour, Health context, Exclusions, Models or Theories (for searches for Models/Theories)

(32)

CIMO

Context, Intervention, Mechanisms, Outcomes (for Realist Syntheses or Management Questions)

(33)

ECLIPSe

Expectations (improvement, innovation or information), Client group (recipients of service), Location (where service is housed), Impact (what change in service and how measured), Professionals involved, Service (for Service Delivery Interventions)

(34)

SPIDER

Sample, Phenomenon of Interest, Design, Evaluation, Research type (for Mixed Methods Reviews)

(35)

 

Search filters

Qualitative search strategies share the intent of other search strategies in aiming to achieve a balance between the sensitivity of not missing relevant items against the specificity of only retrieving the most relevant items whilst, at the same time, keeping the yield within the resources available to complete the review. Search filters may help researchers to achieve this balance.

Search filters have been developed for many of the most commonly used databases (e.g. MEDLINE, Embase, PsycINFO and CINAHL) and these are reported on the ISSG search filters resource. Whilst researchers approach the use of search filters with caution in qualitative evidence synthesis (6), the performance of search filters is acceptable, albeit slightly inferior to search filters studies to identify trials (2).  Interestingly, short filters (e.g. qualitative OR interview* OR Interviews/ OR findings) have, in a limited number of case studies (7, 36), been demonstrated to perform comparably with more extensive filters. Reasons for this observation remain unclear but may relate to less common qualitative terms retrieving records already retrieved by more common terms, impairing their added value and unique yield. Unlike trials there has been no retrospective effort to index qualitative research and the MeSH term for "qualitative research" was only introduced in the year 2003. 

It is important to note that there is no dominant search filter used or recommended in practice to identify qualitative research. Unlike reviews of intervention effectiveness, where the Cochrane HSSS is rightly dominant and widely used, a researcher using a qualitative literature search filter will need to match judiciously their choice of filter to the purpose of the literature search which will, in turn, vary by review (2).

Comprehensive literature searching?

For reviews of intervention effectiveness, a comprehensive, exhaustive literature search for studies is commonly perceived as the default requirement. Researchers have demonstrated that even a single missing study may have a negative impact on the precision of an estimate of intervention effectiveness.

Qualitative evidence synthesis does not offer such a definitive demonstration of the impact of a missing study. A complete Interpretation of a phenomenon of interest may still be possible in the absence of missing studies, particularly if the missing studies share the characteristics ("more of the same") of studies already represented within the sample. The value of comprehensive literature searches is less clear. Further research on how to approach these issues is indicated (37). We note that it may not be possible to undertake ‘exhaustive’ literature searches for qualitative research and the value of doing so continues to be challenged. It is important to understand that when the intent of a qualitative evidence synthesis is interpretive, rather than descriptive, literature searches for qualitative research may not need to be as comprehensive in their coverage as those undertaken to identify quantitative studies (38). On the other hand, unintentional omission of studies capturing particular perspectives, professions or geographical contexts may miss Important nuances reinforcing the point that diversity within the relevant sample may be the most Important driver for the literature search strategy. 

Reporting

Reporting the process of literature searching in qualitative evidence synthesis is no less important than for reviews of intervention effectiveness; more so if alternatives to comprehensive sampling are to be used and justified. With careful and disciplined documentation of the search processes by the searcher, producing a transparent account of literature searching is possible even for a more Iterative process (2). Reporting standards for qualitative evidence synthesis, eMERGe for meta-ethnography (39), RAMESES for meta-narrative approaches (40)(alongside its higher profile guidance for realist syntheses (41)) and ENTREQ for generic approaches to QES (42). All of these include items relating to the documentation of the search process. Most of these reporting standards are derivative from the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) reporting standard and so the number of studies identified by various literature search methods should be reported within a flowchart (43). Reporting of search strategies should also follow generic reporting guidance. While not a reporting standard per se, the EQUATOR Inventory does include a prototypic structure for reporting search strategies, STARLITE (44), compiled from almost 50 published QES:

Sampling strategy

Type of study

Approaches

Range of years

Limits

Inclusion and exclusions

Terms used

Electronic sources

The use of a search narrative has been recommended to improve the transparency of literature searching (45, 46). Whilst this has not been explicitly examined in qualitative literature searching, providing detail on the conceptual and contextual details of the literature searching process is likely to hold value in explaining the conception and execution of the search strategy in a clear and transparent way (46).

 

Reference list

  • (1) Carroll C. Qualitative evidence synthesis to improve implementation of clinical guidelines. BMJ 2017, 356. [Further reference details] [Publication appraisal] [Free full text]
  • (2) Booth A. Searching for qualitative research for inclusion in systematic reviews: a structured methodological review. Syst Rev 2016, 5(1):74. [Further reference details] [Publication appraisal] [Free full text]
  • (3) Centre for Reviews & Dissemination. Systematic reviews: CRD’s guidance for undertaking reviews in healthcare. York: Centre for Reviews and Dissemination, University of York; 2009. [Further reference details] [Publication appraisal] [Free full text]
  • (4) Booth A. Qualitative Evidence Synthesis. In: Facey K. PHH, Single A. (editors). Patient Involvement in Health Technology Assessment.  Singapore: Adis; 2017: 187-199. [Further reference details] [Publication appraisal] [Free full text]
  • (5)  Murphy E, Dingwall R, Greatbatch D, Parker S, Watson P. Qualitative research methods in health technology assessment: a review of the literature. Health technology assessment (Winchester, England) 1998, 2(16):iii-ix, 1-274. [Further reference details] [Publication appraisal] [Free full text]
  • (6) Grant, MJ. How does your searching grow? A survey of search preferences and the use of optimal search strategies in the identification of qualitative research. Health Info Libr J 2004, 21. [Further reference details] [Publication appraisal] [Free full text]
  • (7) Gorecki C, Brown JM, Briggs M, Nixon J. Evaluation of five search strategies in retrieving qualitative patient-reported electronic data on the impact of pressure ulcers on quality of life. J Adv Nurs 2010, 66. [Further reference details] [Publication appraisal] [Free full text]
  • (8)  Toye F, Seers K, Allcock N, Briggs M, Carr E, Barker K. Meta-ethnography 25 years on: challenges and insights for synthesising a large number of qualitative studies. BMC Med Res Methodol 2014, 14. [Further reference details] [Publication appraisal] [Free full text]
  • (9) Shaw RL BA, Sutton A, Miller T, Smith J, Young B. Finding qualitative research: an evaluation of search strategies. BMC Med Res Methodol. 2004;4:5. BMC Med Res Methodol 2004, 4(5). [Further reference details] [Publication appraisal] [Free full text]
  • (10) Evans D. Database searches for qualitative research. J Med Libr Assoc 2002, 90. [Further reference details] [Publication appraisal] [Free full text]
  • (11) Campbell R, Pound P, Pope C, Britten N, Pill R, Morgan M. Evaluating meta-ethnography: a synthesis of qualitative research on lay experiences of diabetes and diabetes care. Soc Sci Med 2003, 56. [Further reference details] [Publication appraisal] [Free full text]
  • (12) Dixon-Woods M, Fitzpatrick R, Roberts K. Including qualitative research in systematic reviews: opportunities and problems. J Eval Clin Pract 2000, 7. [Further reference details] [Publication appraisal] [Free full text]
  • (13) Atkins S, Lewin S, Smith H, Engel M, Fretheim A, Volmink J. Conducting a meta-ethnography of qualitative literature: lessons learnt. BMC Med Res Methodol 2008, 8. [Further reference details] [Publication appraisal] [Free full text]
  • (14) Noyes J PJ, Pearson A, Hannes K, Booth A.  Chapter 20: Qualitative research and Cochrane reviews. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.handbook.cochrane.org. [Further reference details] [Publication appraisal] [Free full text]
  • (15) Wright K, Golder S, Lewis-Light K. What value is the CINAHL database when searching for systematic reviews of qualitative studies? Syst Rev 2015, 4. [Further reference details] [Publication appraisal] [Free full text]
  • (16) Hannes K, Macaitis K. A move to more systematic and transparent approaches in qualitative evidence synthesis: update on a review of published papers. Qual Res 2012, 12(4):402-442. [Further reference details] [Publication appraisal] [Free full text]
  • (17) Dixon-Woods M, Booth A, Sutton AJ. Synthesizing qualitative research: a review of published reports. Qual Res 2007, 7(3):375-422. [Further reference details] [Publication appraisal] [Free full text]
  • (18) Subirana M, Sola I, Garcia JM, Gich I, Urrutia G. A nursing qualitative systematic review required MEDLINE and CINAHL for study identification. J Clin Epidemiol 2005, 58(1):20-25. [Further reference details] [Publication appraisal] [Free full text]
  • (19) DeJean D, Giacomini M, Simeonov D, Smith A. Finding Qualitative Research Evidence for Health Technology Assessment. Qual Health Res 2016, 26(10):1307-1317. [Further reference details] [Publication appraisal] [Free full text]
  • (20) Booth A: Chapter 3: Searching for Studies. In: Noyes J BA, Hannes K, Harden A, Harris J, Lewin S, Lockwood C. (editors). Supplementary Guidance for Inclusion of Qualitative Research in Cochrane Systematic Reviews of Interventions. Cochrane Collaboration Qualitative Methods Group; 2011. [Further reference details] [Publication appraisal] [Free full text]
  • (21) Cooper C, Booth A, Britten N, Garside R. A comparison of results of empirical studies of supplementary search techniques and recommendations in review methodology handbooks: a methodological review. Syst Rev 2017, 6:234. [Further reference details] [Publication appraisal] [Free full text]
  • (22) Cooper C, Lovell R, Husk K, Booth A, Garside R. Supplementary search methods were more effective and offered better value than bibliographic database searching: A case study from public health and environmental enhancement. Res Synth Methods 2018, 9(2):195-223. [Further reference details] [Publication appraisal] [Free full text]
  • (23) Mackay G. Is there a need to differentiate between qualitative and quantitative searching strategies for literature reviewing? Qual Soc Work 2007, 6(2):231-241. [Further reference details] [Publication appraisal] [Free full text]
  • (24) Cooper C, Booth A, Varley-Campbell J, Britten N, Garside R. Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies. BMC Med Res Methodol 2018, 18(1):85. [Further reference details] [Publication appraisal] [Free full text]
  • (25) Sandelowski M, Barroso J. Searching for and retrieving qualitative research reports. In: Sandelowski M, Barroso J. (editors). Handbook for synthesizing qualitative research. New York: Springer; 2007. [Further reference details] [Publication appraisal] [Free full text]
  • (26) Stern C, Jordan Z, McArthur A. Developing the review question and inclusion criteria. AJN: Am J Nurs 2014 Apr;114(4):53-6. [Further reference details] [Publication appraisal] [Free full text]
  • (27) Petticrew M, Roberts H. How to find the studies: the literature search. In: Petticrew M, Roberts H. (editors). Systematic reviews in the social sciences: A practical guide. Oxford: Blackwell Publishing; 2006. [Further reference details] [Publication appraisal] [Free full text]
  • (28) Booth A. Australian supermodel? A practical example of evidence-based library and information practice. (EBLIP). Health Info Libr J 2006 Mar;23(1):69-72. [Further reference details] [Publication appraisal] [Free full text]
  • (29) Richardson WS, Wilson MC, Nishikawa J, Hayward RS. The well-built clinical question: a key to evidence-based decisions. ACP J Club 1995, 123. [Further reference details] [Publication appraisal] [Free full text]
  • (30) Howell Major C S-BM. Designing the synthesis   In: Major CH, Savin-Baden, M. (editors). An introduction to qualitative research synthesis: managing the information explosion in social science research. London: Routledge; 2010: 43–55. [Further reference details] [Publication appraisal] [Free full text]
  • (31) Methley AM, Campbell S, Chew-Graham C, McNally R, Cheraghi-Sohi S. PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv Res 2014, 14. [Further reference details] [Publication appraisal] [Free full text]
  • (32) Booth A, Carroll C. Systematic searching for theory to inform systematic reviews: is it feasible? Is it desirable? Health Info Libr J 2015, 32. [Further reference details] [Publication appraisal] [Free full text]
  • (33) Denyer D, Tranfield D. Producing a systematic review. In: Buchanan DA, Bryman A. (editors). The SAGE handbook of organizational research methods. London: SAGE Publications Ltd; 2009. [Further reference details] [Publication appraisal] [Free full text]
  • (34) Wildridge V, Bell L. How CLIP became ECLIPSE: a mnemonic to assist in searching for health policy/management information. Health Info Libr J 2002, 19. [Further reference details] [Publication appraisal] [Free full text]
  • (35) Cooke A, Smith D, Booth A. The SPIDER tool for qualitative evidence synthesis. Qual Health Res 2012, 22. [Further reference details] [Publication appraisal] [Free full text]
  • (36) Flemming K, Briggs M. Electronic searching to locate qualitative research: evaluation of three strategies. J Adv Nurs 2007, 57. [Further reference details] [Publication appraisal] [Free full text]
  • (37) Benoot C, Hannes K, Bilsen J. The use of purposeful sampling in a qualitative evidence synthesis: A worked example on sexual adjustment to a cancer trajectory. BMC Med Res Methodol 2016, 16:21. [Further reference details] [Publication appraisal] [Free full text]
  • (38) Booth A. Cochrane or cock-eyed? How should we conduct systematic reviews of qualitative research?  In: Qualitative Evidence-based Practice Conference, Taking a Critical Stance. Coventry University; 2001. [Further reference details] [Publication appraisal] [Free full text]
  • (39) MERGe:Meta-ethnography Reporting Guidance (http://emergeproject.org/) [Further reference details] [Publication appraisal] [Free full text]
  • (40) Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. RAMESES  publication standards: meta-narrative reviews. BMC Medicine 2013, 11(1):20. [Further reference details] [Publication appraisal] [Free full text]
  • (41) Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. RAMESES publication standards: realist syntheses. BMC Medicine 2013, 11(1):21. [Further reference details] [Publication appraisal] [Free full text]
  • (42) Tong A, Flemming K, McInnes E, Oliver S, Craig J. Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Med Res Methodol 2012, 12(1):181. [Further reference details] [Publication appraisal] [Free full text]
  • (43) Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLOS Medicine 2009, 6(7):e1000097. [Further reference details] [Publication appraisal] [Free full text]
  • (44) Booth A. "Brimful of STARLITE": toward standards for reporting literature searches. J Med Libr Assoc 2006, 94. [Further reference details] [Publication appraisal] [Free full text]
  • (45) Craven J, Levay P. Recording database searches for systematic reviews - What is the value of adding a narrative to peer-review checklists? A case study of NICE interventional procedures guidance. Evidence Based Library and Information Practice 2011, 6(4):72-87. [Further reference details] [Publication appraisal] [Free full text]
  • (46) Cooper C, Dawson S, Peters J, Varley-Campbell J, Cockcroft E, Hendon J, Churchill R. Revisiting the need for a literature search narrative: A brief methodological note. Res Synth Methods 2018 Sep;9(3):361-365. [Further reference details] [Publication appraisal] [Free full text]