Organizational aspects

Last revised: 
2017-10-30

Introduction

Health Technology Assessments (HTAs) not only evaluate a health technology and its effectiveness (and often cost-effectiveness) but also consider organizational aspects surrounding its implentation, or sometimes removal, within a specific context or setting. This domain of an HTA examines how various types of resources (administrative, human, technological, etc.) need to be structured when implementing a technology. Any impacts that may result within the health care organization or the health system as a whole are considered. (1,2).

In general, the organizational domain explored the following issues, but may also consider others (1,2):

  1. Health care delivery processes and how the technology may affect current work flows
  2. The structure of health care services and equitable access to the new technology
  3. Process related costs for purchasing and setting up the new technology along with budget impacts
  4. Management issues
  5. Cultural issues including acceptance of the new technology by those within health care organizations

General Search Guidance

There is little information regarding the optimal methods for conducting analyses in this domain and consequently little guidance on best practices for searching the evidence base.  EUnetHTA’s HTA Core Model (1) and the Danish Centre for Health Technology Assessment’s Health Technology Assessment Handbook (2) offer the most detailed guidance in this area.

In general, both sources agree that this is a challenging area for information retrieval as evidence on the organization and delivery of health services encompases a wide range of disciplines, study types, and is spread across a wide range of published and grey literature.  The information required for this section of an HTA  is often context (and often country) specific which can result in little to no published literature being available. (1,2)

It is recommended that, as a first attempt, an extensive literature search focusing on identifyiing systematic reviews of organizational aspects should be conducted. If no systematic reviews are available, then the search should be revised to focus on guidelines and relevant primary studies. If no relevant data are identified, the third step is to identify primary data which might involve conducting surveys or interviews of healthcare professionals and content experts.  Data might also be obtained from administrative databases of the relevant organizations involved in the analysis (1). New primary qualitative research might be the only way to assess real world practice use and misuse. (2)

Sources to search

A wide range of sources of published and unpublished (grey) information should be searched. Other search techniques should also be considered including contacting experts and scanning reference lists of relevant papers and hand searching of journals.  Information should be gathered not just from traditional health sciences literature sources but also from sources of social sciences, business, and even education literature.  The choice and number of resources to search will depend on the topic of the assessment and the time/resources available for searching. At a minimum, the most commonly used databases below should be consulted (1,2).

Resources recommended to search for the organizational domain (1,2) include:

  • MEDLINE/Pubmed
  • CRD (Centre for Reviews and Dissemination) Databases
  • DARE (Database of Abstracts of Reviews of Effects)
  • HTA (Health Technology Assessment)
  • NHS EED (National Institute for Health Research / Economic Evaluation Database)
  • CINAHL (Cumulative Index to Nursing and Allied Health Literature)
  • Cochrane Library
  • GIN (Guideline International Network)

In addition the following sources could also be searched:

  • Health sciences databases (Embase, PsycINFO, any regional/country specific databases of clinical studies such as HMIC from the U.K.)
  • Social sciences databases (Sociological Abstracts, Social Services Abstracts, Social Care Online, SocINDEX, ASSIA)
  • Education databases (ERIC)
  • Ongoing research databases (EUnetHTA POP database, ClinicalTrials.gov, Prospero)
  • Horizon scanning databases (EuroScan, ukBIOSIS)
  • Other databases (Joanna Briggs Institute Library, EUnetHTA pool of structured HTA information)
  • Grey literature (dissertations, guidelines, conference proceedings, OAIster, registries and statistics, administrative data, pharmaceutical registries, working papers, industry reports, handbooks, organizational reports).

 

Types of Research Studies to Include in the Search

Because of the complexity of the organizational domain in terms of the variety of literature that is needed, no single type of research study is appropriate. The evidence base may encompass many varying types of studies, both quantitative and qualitative.  The HTA Core Model (1) recommends searching  for the following study types:

  • Health delivery process questions: Guidelines, observational, mostly qualitative, partly quantitative, RCT or systematic reviews of RCTs
  • Structure of health care questions: Guidelines, observational, mostly qualitative.
  • Process related costs questions: Guidelines, producer technical handbooks, Costing and budget impact analyses
  • Management questions: Guidelines, observational studies mostly qualitative, consensus, protocols
  • Culture questions: Observational, mostly qualitative.

With this is mind, we also recommend that individuals consult the chapter (note: in development) on searching for qualitative literature to aid in the development of search strategies for the organizational domain.  


Designing Search Strategies

In terms of designing the search strategy for this topic area, there is little guidance available and we suggest that information specialists explore published reviews and HTAs to see how others have searched for topics such as health delivery processes and health structures.

Search Filters

A number of search filters can be employed in the design of search strategies, each with a specific focus.  Wilczynski et al. (3) have developed a health services research filter and Simon et Al. (4) have created a filter aimed at uncovering nurse staffing research.  Van Walraven et al. (5) have developed a filter to identify studies that use administrative data, and Hempel et al. on quality improvement interventions (6).  Each of these authors acknowledge that research in these topic areas is difficult to search for because of the wide variety of applicable subject headings/terms and the variable keywords and language used to describe the field (3-6). While all of these filters have quite good sensitivity, they all have much poorer precision which is likely to result in quite large search yields and many irrelevant records that need to be manually screened.

Additional search filters can be identified from the InterTASC Information Specialists' Sub-Group (ISSG) Search Filter Resource.

 

Reference list

  • (1) EUnetHTA Joint Action 2 Work Package 8. HTA Core Model ® version 3.0 (Pdf). 2016. [Further reference details] [Publication appraisal] [Free full text]
  • (2) Kristensen F, Sigmund H. (eds.) Health Technology Assessment Handbook. Copenhagen: Danish Centre for Health Technology Assessment, National Board of Health; 2007. [Further reference details] [Publication appraisal] [Free full text]
  • (3) Wilczynski NL, Haynes RB, Lavis JN, Ramkissoonsingh R, Arnold-Oatley AE. Optimal search strategies for detecting health services research studies in MEDLINE. CMAJ. 2004;171(10):1179-85. [Further reference details] [Publication appraisal] [Free full text]
  • (4) Simon M, Hausner E, Klaus SF, Dunton NE. Identifying nurse staffing research in Medline: development and testing of empirically derived search strategies with the PubMed interface. BMC Med Res Methodol. 2010;10:76. [Further reference details] [Publication appraisal] [Free full text]
  • (5) Van Walraven C, Bennett C, Forster AJ. Derivation and validation of a MEDLINE search strategy for research studies that use administrative data. Health Serv Res. 2010;45(6 Pt 1):1836-45. [Further reference details] [Publication appraisal] [Free full text]
  • (6) Hempel S, Rubenstein LV, Shanman RM, Foy R, Golder S, Danz M, et al. Identifying quality improvement intervention publications - a comparison of electronic search strategies. Implement Sci. 2011;6:85. [Further reference details] [Publication appraisal] [Free full text]