Databases

Scope: provide the database name(s); add to records where a database rather than a website or interface is discussed

Appraisal of: Droste S, Dintsios CM, Gerber A. Information on ethical issues in health technology assessment: How and where to find them. Int J Technol Assess Health Care 2010; 26(4): 441–449.

Short description: 

The aim of the article is to present a procedure to identify information on ethical implications of the use and value-related decision-making in the assessment of health technologies. The purpose of the project is to develop, propose and test a step-by step workflow adapted to the specific issues of information on ethical issues (searching for arguments and qualitative data).

The article provides information on the background of developing the methodological approach as well as on the 8 working steps - Step 1 “Translation of the search question using the PICO scheme and additional components” to Step 8 “Final quality check and calculation of precision and recall”. The article is supplemented by the application results of the example “Ethical issues of autologous stem cell transplantation in metastatic breast cancer”.

Limitations stated by the author(s): 
  • Not all ethical issues related to health technologies can be identified by means of the approach described.
  • Some ethical issues cannot be found with systematic approaches - additional non-systematic searching may be useful.
  • The ethical issues relevant to the health technology investigated should be known in more detail before starting the systematic information retrieval.
  • Alleged innovations cannot be identified as such by the methodological approach.
Limitations stated by the reviewer(s): 
- Reference(s) to the validation work could have been provided. - The authors claim in the objective that if choosing the proper methodological approach: the central ethical dimensions then should be answerable. This seems like a too strong claim, what the authors do is to provide a brilliant methodological approach to address central ethical dimensions, to the extent there is anything published concerning these. Whether that answers ethical concerns is to some extent another matter and might call for a separate ethics analysis. Both since the sources might give conflicting answers, or give no answers at all. - Lack references to the database Philosopher’s index which in some cases might provide sources for a more general but relevant ethical discussion.
Study Type: 
Single study

Appraisal of: Rauprich O, Nolte M, Vollmann J. Systematische Literaturrecherchen in den Datenbanken PubMed® und BELIT: Ein Werkstattbericht. Ethik Med 2010; 22(1): 59–67.

Short description: 

The aim of the article is to present the systematic searching for Medical Ethics literature in the bibliographic databases BELIT (German Reference Centre for Ethics in the Life Sciences, DRZE) and PubMed (National Library of Medicine, NLM) by means of two examples. The purpose of the project is to evaluate the feasibility and capacity of BELIT and PubMed as well as to design optimized search strategies for these two databases.

The article provides information on the selected databases, the search strategies designed for each source, and on the comparative analysis of the search results. Subjects of application are the examples “Concept of health and disease” and “Ethical, legal, social and economic issues of financing fertility treatments”.

In both examples the searches yielded in several hundred hits each with a small overlap of 3 and 4% between BELIT and PubMed. The best results are achieved by combining controlled terms and free-text queries. Using different query languages (German and English) in BELIT further improves the result. While in PubMed building of complex search strategies is possible, BELIT is restricted to process simple search queries.

Limitations stated by the author(s): 

Due to the high effort needed to investigate the study is restricted to PubMed and BELIT. Other databases with information on ethical issues are neglected – although important for systematic searches.

Limitations stated by the reviewer(s): 
This journal article represents a short report version of the study. To fully understand the methods used it is necessary to study the long report version.
Study Type: 
Single study

Appraisal of: Dracos A. SIBIL: uno strumento italiano per il reperimento dell'informazione in bioetica. Ann Ist Super Sanita 2004; 40 (): 283–6.

Short description: 

The aim of the article is to present the development and structure of the project SIBIL (Sistema Informativo per la Bioetica In Linea) provided by the Istituto Superiore di Sanità (ISS), Rome, Italy. The purpose of the project is to collect, index and disseminate the Italian literature on bioethics since 1995.

The article provides information on the background of the database development, the development of the indexing system, the Italian Bioethics Thesaurus (TIB), and the presented publications. These are books, book contributions, journal articles, legal documents, ethical codes, statements, guidelines reports, grey literature, proceedings, electronic documents and resources. Further, information on conferences in bioethics is announced via the SIBIL web page (http://www.iss.it/sibi/index.php?lang=2).

This publication is in Italian. Abstract is provided in English.

 

Limitations stated by the author(s): 

No limitations were stated.

Limitations stated by the reviewer(s): 
No limitations detected by the reviewers. SIBIL is currently searchable via the portal ethicsweb, the European information and documentation system for ethics and science (http://ethicsweb.org/portal).
Study Type: 
Single study

Appraisal of: Petrova M, Sutcliffe P, Fulford KW, Dale J. Search terms and a validated brief search filter to retrieve publications on health-related values in Medline: A word frequency analysis study. J Am Med Inform Assoc 2012; 19(3): 479–488.

Short description: 

The aim of the article is to present search terms (keywords) and a “brief” search filter to identify information on health-related values in Medline developed by using word frequency analysis. The purpose of the project is to support broad-scoping searches by “a brief search filter (≤20 lines) of high precision (≥67%) and acceptable sensitivity (≥67%) which can be used […] in generic medical and health databases (e.g., Medline, Embase, or Cinahl) and across a range of topics (e.g., health conditions, health settings, health interactions, etc.)”.

The article provides extensive information on the background as on health-related values defined by the authors and on word frequency analyses. The application of this analysis in the conditions diabetes, dementia, schizophrenia and obesity resulted in a small overlap of relevant MeSH and free-text search terms (9 of 124 different MeSH terms (7.3%) and 4 of 144 different words (2.8%) were identified in all 4 conditions among the 50 “best” MeSH respective words in each condition). Applying the developed search filter in hypersensitivity and dentistry resulted in (authors defined) sensitivity of 70.1% (hypersensitivity) and 47.1% (dentistry), and precision of 63.6% (hypersensitivity) and 82.6% (dentistry). 

Limitations stated by the author(s): 
  • A number of ‘subjective’ additions to the objectively derived filter improved sensitivity and precision.
  • The brief values filter can be applied productively to other topics in Medline, but there is instability in its performance.
  • The study covers a limited period, limited number of topics, a single database, and a single approach to compiling its corpus of citations.
  • The study lacked well-articulated criteria for distinguishing cases (of values publications) from non-cases. The criteria used may have been overly inclusive.
  • This study did not have the capacity for both theoretical and empirical work of adequate depth.
  • The reported frequencies for text words did not necessarily reflect the number of abstracts in which a word appears.
  • Data cleaning was performed on text words that had a 100% precision, but not for the remainder.
  • Limit to the very field of designing objective search strategies on the basis of word frequency analysis: The field may be lagging behind advances in medical informatics. Some of the methods applied in this study may need to be superseded, including methods that have improved on standard practices of objective search strategy design.

 

 

Limitations stated by the reviewer(s): 
The theoretical concept of values defined by the authors is not fully clear and different from those used in ethical analysis in HTA. Besides the many limitations already stated by the authors there are some significant methodological shortcomings, and the results don’t fulfill the needs of systematic reviews in HTA regarding sensitivity in a sufficient manner. The proposed search filter is clearly not this universal solution as claimed in the authors’ objectives. As well the results indicate that the method of word frequency analysis itself fails in the field of values. The shortcomings in detail: The glossary definitions are not always identical with the common ones in Information Retrieval. Authors’ objective: No reasoning for the line number (and no meaning on the length of each line) and sensitivity and precision. The validation, in particular the external validation, applied by the authors does not fulfill criteria of an evidence-based validation. Building-up the derivation dataset is not reported in a transparent and reproducible manner. As the conditions were just defined by one MeSH term each the sensitivity of the results may be low. Calculating sensitivity of the search filter by using these datasets seems to be questionable (and a self-fulfilling prophecy).
Study Type: 
Single study

Appraisal of: Wilczynski NL, Morgan D, Haynes RB; Hedges Team. An overview of the design and methods for retrieving high-quality studies for clinical care. BMC Med Inform Decis Mak. 2005 Jun 21;5:20.

Reviewer(s): 
Short description: 

The authors describe the methods they use to develop optimal search filters (hedges) to identify a range of study types: causation, prognosis, diagnosis, treatment, economics, clinical prediction guides, reviews, costs, and qualitative:  the filters were designed for MEDLINE and Embase. The authors built a large gold standard (reference set) by hand searching 170 journals for one year: 2000.  Relevant records were defined and were selected to represent best research methods. The gold standard records were then downloaded from  MEDLINE, Embase, CINAHL, and PsycINFO with the subject indexing assigned by each database.  Candidate search terms were identified from the gold standard records and consulting experts. The sensitivity, specificity, precision, and accuracy of unique search terms and combinations of search terms were calculated. Once the performance parameters of individual search terms were computed, the authors  selected individual terms for the construction of search strategies by choosing search terms with specific levels of sensitivity and specificity (which varied by database). The authors also used logistic regression to explore ways to improve filter performance. Strategies were developed in a random selection of 60% of the gold standard and validated in the remaining 40%. No statistical differences in performance were found between the two strategy development methods or between the test and validation results, so the majority of filter development used the Boolean approach and search strategies were developed using all records in the database.

The gold standard database numbered 60,330 records, each with up to 11 data fields. Filters were developed for studies of causation, prognosis, diagnosis, treatment, economics, clinical prediction guides, reviews, costs, and those of a qualitative nature.

Limitations stated by the author(s): 

The authors do not state any limitations to their methods.

Limitations stated by the reviewer(s): 
The journals contributing to the gold standard were largely general (internal medicine, general medical practice, mental health and general nursing practice) rather than specialty focused. There may be issues about generalizability to the latter journals. Records were selected for the gold standard according to the authors’ definition of the specific topics and best study designs for those topics. These definitions need to be compared to the definitions of any user. The authors decided, following testing, to use all of the database records to both develop and test the filters. Ideally validation data on a different reference standard should be provided.
Study Type: 
Single study

Appraisal of: Zechmeister-Koss I, Schnell-Inderst P, Zauner G. Appropriate evidence sources for populating decision analytic models within health technology assessment (HTA): a systematic review of HTA manuals and health economic guidelines. Med Decis Mak

Short description: 

This was a systematic review of HTA manuals and health economic guidelines focused on identifying search approaches and information sources to populate decision analytic models. 28 documents in English and German were identified. The overall messages from these guidelines were that searches should be as systematic as possible in terms of being transparent, but that only some model parameters, chiefly effects evidence, need by informed by systematic reviews. A large number of data sources of varying types may provide information for the parameters, but there seems to be little consensus about which are essential.

There are two online supplements. One lists excluded guidelines and the second provides some detail on each of the included guidelines.

Limitations stated by the author(s): 

The search was restricted to publications in English and German and to Western countries. The focus was on information to inform model parameters rather than other aspects of developing decision models. The information is generic rather than focusing on specific model approaches.

Limitations stated by the reviewer(s): 
The resources listed in individual guidelines are not presented. The search approaches suggested in the individual guidelines are not summarized in detail.
Study Type: 
Single study
Related Chapters: 

Appraisal of: Shemilt I, Mugford M, Vale L, Craig D. Searching NHS EED and HEED to inform development of economic commentary for Cochrane intervention reviews. Oxford: Cochrane Collaboration; 2011.

Short description: 

This study explored methods to incorporate economic evaluations into Cochrane reviews, and part of the study involved searching NHS EED and HEED for economic studies. NHS EED and HEED searches were developed to identify relevant economic studies for 35 Cochrane reviews. The authors reported on database overlap and unique yield: 51 studies were unique to NHS EED and 9 to HEED. The authors recommend that searchers should use the strategies designed to find the review population (from the effects searches) when searching for burden of illness or cost of illness studies in NHS EED and HEED. If searching for economic evaluations, the intervention search strategy from the review is recommended. Searching both NHS EED and HEED is required to identify all potentially relevant economic evaluations, but only searches of HEED are required for other economic studies after changes to NHS EED in 2012.

Limitations stated by the author(s): 

No limitations are described.

Limitations stated by the reviewer(s): 
The authors seem to have re-used the population and intervention strategies from within the Cochrane reviews that they sampled. The authors did not assess the quality of those strategies, so there is the potential that the best strategies were not used in NHS EED and HEED. In addition, the two databases have very different interfaces and the same strategies would not have been feasible to test. The authors also make no mention of the importance of searching other databases, such as MEDLINE or Embase, when looking for economics-related material, especially for recently published information.
Study Type: 
Single study
Related Chapters: 

Appraisal of: Dakin H. Review of studies mapping from quality of life or clinical measures to EQ-5D: an online database. Health and Quality of Life Outcomes 2013;11(1):151.

Reviewer(s): 
Short description: 
This paper reported the creation of a database of mapping studies. The mapping studies provided mapping algorithms to convert quality of life instrument values to EQ-5D (EuroQoL) instrument values. Following a search of MEDLINE and three websites, the authors identified ninety mapping studies, which reported eighty mapping algorithms. These were described in the paper. The studies can be searched in a free internet database: http://www.herc.ox.ac.uk/downloads/mappingdatabase. The database also contains references to studies validating mapping algorithms and supplementary information including tools to generate predictions. It is intended that the database will be updated. It is possible to register by email to be informed of updates to the database.
Limitations stated by the author(s): 
The author did not search Embase or the grey literature where mapping may also be reported. Some studies might have been excluded because of lack of information but might have proved to be relevant if the authors had been contacted to provide more information.
Limitations stated by the reviewer(s): 
The author did not report the full search terms and search term combinations used to identify studies from MEDLINE, so it is difficult to know whether the search was fit for purpose.
Study Type: 
Single study
Related Chapters: 

Appraisal of: Waffenschmidt S, Hausner E, Engel L, Volz F, Kaiser T. Benefit of searching different databases to identify health economic evaluations in German HTA-reports. Abstract presented at: Health Technology Assessment International (HTAi) 7th Annua

Reviewer(s): 
Short description: 
This study assessed the benefits of searching various databases used for identifying health economic evaluations. German language HTA reports that incorporated economic evaluations were consulted. The data sources used in the literature search were analyzed to see where relevant references were retrieved from (standard international, German or health economics databases). The findings showed that 90% of citations were retrieved from searches of MEDLINE, Embase and the Cochrane Library (HTA database). Neither German databases nor health economics databases (such as NHS EED) provided any unique citations. These findings indicate that German HTA reports identify most of their relevant health economics citations through standard international databases. However, if a comprehensive pool of studies is required, searching additional sources beyond bibliographic databases, including hand searching, seems to be required.
Limitations stated by the author(s): 
This is a presentation of preliminary findings.
Limitations stated by the reviewer(s): 
Since results are available only in abstract/poster form, complete study information is not available. The study was limited to German language reports, so results may not be transferable to other countries or languages. We note that health economics databases tend to be populated from MEDLINE or Embase and are therefore unlikely to yield unique references: their value lies in being time saving and value-added.
Study Type: 
Single study
Related Chapters: 

Appraisal of: Meier BM, Cabrera OA, Ayala A, Gostin LO. Bridging international law and rights-based litigation: Mapping health-related rights through the development of the Global Health and Human Rights Database. Health Hum Rights 2012;14(1):20-35.

Short description: 
The aim of the article is to present the development process of the Global Health and Human Rights Database provided by O’Neill Institute for National and Global Health Law, the World Health Organization (WHO) and Lawyers Collective. The purpose of the database is to bring together judgments, international and regional legal instruments, and national constitutions involving health and human rights. The article provides information about the background for the database development, the development process, organization of the content, and search functionalities.
Limitations stated by the author(s): 
No limitations stated by the authors.
Limitations stated by the reviewer(s): 
No limitations detected by the reviewers.
Study Type: 
Single study
Related Chapters: 
Syndicate content