Diagnostic accuracy

Last revised: 


HTA may include assessment of new diagnostic technologies or techniques. These can involve the identification and review of diagnostic test accuracy (DTA) studies designed to differentiate between individuals with and without a target condition (1). The Cochrane Collaboration has published an evidence-based guide to searching for DTA studies, which provides the basis for this summary (2).

DTA studies tend to be poorly reported and searching for them can be problematic due to this inadequate reporting and inconsistent terminology, the absence of appropriate indexing terms in some databases for this publication type, and inconsistent use of suitable indexing terms where they are available. (2).

Sources to search

Relying only on searching MEDLINE is not recommended, as it is unlikely to be the most comprehensive source of diagnostic information and because diagnostic studies are not easy to retrieve efficiently in bibliographic databases (3). Relative recall analysis of systematic reviews has also suggested other databases might yield additional studies including Science Citation Index, BIOSIS and LILACS (3). However, two recent analyses have suggested that fewer databases might be adequate.  An analysis of ten meta-analyses found that only using studies indexed in MEDLINE did not impact significantly on the sensitivity and specificity estimates of the meta-analyses in those reviews (4). However, this research was based on known-item  searches of MEDLINE: review searches may not detect all the records in MEDLINE that might be relevant to a review, so searching other databases provides opportunities to pick up (MEDLINE indexed) studies by other routes.  Another recent study of nine reviews performed by a single research group found that the reviewers’ original searches would have found 85% of the their included studies from MEDLINE and Embase (range: 60-100%)(5).  Adding reference checking to the process would have found 93% of the included studies. The Cochrane Handbook (2), based on available research-evidence, currently recommends that searches should include the following  databases for reviews and primary studies:

●      MEDLINE
●      Embase
●      ARIF
●      HTA database
●      DARE (closed to new records from March 2015)
●      Cochrane Database of Systematic Reviews
●      Searches for unpublished studies in dissertations databases and grey literature databases
●      Reference checking
●      Citation searches in services such as Science Citation Index, Scopus, and Google Scholar, as well as related
         articles options in interfaces such as PubMed or Ovid.

In addition the following new databases could also be searched:

●     PROSPERO - register of systematic reviews
●     Epistemonikos - collection of systematic reviews and their included studies
●     PDQ Evidence - collection of systematic reviews about health systems and their included studies 

HTA agencies may undertake assessments of diagnostic tests and so agency websites should also be explored, for example NICE diagnostic test guidance can be accessed here.

Although the proportion of ongoing studies investigating diagnostic test accuracy may still be relatively low (6), some are being recorded prospectively on trials registers such as ClinicalTrials.gov and the ICTRP portal.

The evidence for the value of handsearching is currently sparse, with one recent study of one topic showing that handsearching contributed little (7). It is possible that the topic of the research was well defined and the database searches were exemplary, and therefore the handsearching contribution would be different in other topics (7). More evidence is required on the yield and value of handsearching. Where a topic is publihsed in journals that are not indexed in bibliographic databases, handsearching can still serve a purpose but this needs to be evaluated question by question.

Designing search strategies

Search strategies should be designed to be highly sensitive using a wide variety of search terms, both text words and subject indexing, to ensure that the many different ways that a test may be described feature in the search (2). The search should reflect some, but not necessarily all, of the key concepts of the review (2). The search is likely to capture the index test being investigated and the target condition being diagnosed (2). A third set of terms can be considered to capture the patient description or the reference standard. The development of search strategies for DTA studies can be challenging and may involve several iterations to reach a strategy that captures the complex way records may present concepts of diagnosis (2). Cochrane Reviews of diagnostic test accuracy studies and the Cochrane handbook provide examples of search approaches for these, often complex, topics. Strategies may include both general terms (such as the generic type of diagnostic method, for example dipsticks) and specific terms such as named dipstick tests (2).

There are many methodological search filters designed to capture studies of diagnostic test accuracy and including test measurement terms such as sensitivity and accuracy (8). The evidence, however, on the performance of DTA search filters suggests that combining filters with a search for a population and an index test is likely to miss relevant studies (9,10,11). Search filters for DTA studies do not seem to perform consistently and may result in unacceptable reductions in sensitivity (9, 10, 11,12). However, as long as DTA filters are not the only approach, they may be useful as one component of a search strategy which involves several search approaches:  a “multi-stranded” approach involves multiple queries run sequentially and using different combinations of concepts. Search filters can be identified from the InterTASC Information Specialists' Sub-Group (ISSG) Search Filter Resource.

Subheadings (floating subheadings and subheadings attached to the index test or the target condition) may be a helpful component of the search strategy (2).

Developing a search strategy can be iterative and complex and it can be helpful to have topic experts to review samples of search results for relevance and it is always helpful to be able to test retrieval against sets of known relevant records.

The revised and updated searching chapter of the Cochrane Handbook (2) is in preparation. To avoid duplication of effort with the development of the Cochrane Handbook, appraisals have only been prepared for more recent studies in this chapter. Once the revised Cochrane Handbook is available it will be used to update this chapter further.

Reference list

● (1) EUnetHTAWork Package 4. HTA Core Model® for Diagnostic Technologies v. 1.0r; 2008. [Further reference details] [Publication appraisal] [ Free Full text ]
● (2) de Vet H C W, Eisinga A, Riphagen I I, Aertgeerts B, Pewsner D. Searching for studies. In Cochrane handbook for systematic reviews of diagnostic test accuracy. Version 0.4 ed. The Cochrane Collaboration, 2008. [Further reference details] [Publication appraisal] [Free Full text]
● (3) Whiting P, Westwood M, Burke M, Sterne J, Glanville J. Systematic reviews of test accuracy should search a range of databases to identify primary studies. J Clin Epidemiol 2008;61(4): 357.e1-357.e10. [Further reference details ] [Publication appraisal] [Free Full text]
●(4) van Enst WA, Scholten RJ, Whiting P, Zwinderman AH, Hooft L. Meta-epidemiologic analysis indicates that MEDLINE searches are sufficient for diagnostic test accuracy systematic reviews. J Clin Epidemiol 2014;67(11): 1192-1199. [Further reference details] [Publication appraisal] [Free Full text]
● (5) Preston L, Carroll C, Gardois P, Paisley S, Kaltenthaler E. Improving search efficiency for systematic reviews of diagnostic test accuracy: an exploratory study to assess the viability of limiting to MEDLINE, EMBASE and reference checking. Syst Rev. 2015;4: 82. [Further reference details] [Publication appraisal] [Free full text]
● (6) Korevaar DA, Bossuyt PMM, Hooft L. Infrequent and incomplete registration of test accuracy studies: analysis of recent study reports. BMJ Open. 2014;4: e004596. [Further reference details] [Publication appraisal] [Free full text]
●(7) Glanville J, Cikalo M, Crawford F, Dozier M, McIntosh H. Handsearching did not yield additional unique FDG-PET diagnostic test accuracy studies compared with electronic searches: a preliminary investigation. Res Synth Methods 2012;3(3): 202-213. [Further reference details] [Publication appraisal] [Free Full text]
●(8) ISSG Search Filter Resource [web page]. York: Centre for Reviews and Dissemination and InterTASC Information Specialists SubGroup; 2013. [Further reference details] [Publication appraisal] [Free Full text ]
●(9) Leeflang M M G, Scholten R J, Rutjes A W S, Reitsma J B, Bossuyt P M. Use of methodological search filters to identify diagnostic accuracy studies can lead to the omission of relevant studies. J Clin Epidemiol 2006;59(3): 234-240. [Further reference details] [Publication appraisal] [Free Full text]
● (10) Ritchie G, Glanville J, Lefebvre C. Do published search filters to identify diagnostic test accuracy studies perform adequately? Health Info Libr J 2007;24(3):188–192. [Further reference details] [Publication appraisal] [Free Full text ]
● (11) Whiting P, Westwood M, Beynon R, Burke M, Sterne J A, Glanville J. Inclusion of methodological filters in searches for diagnostic test accuracy studies misses relevant studies. J Clin Epidemiol 2011;64(6): 602-607. [Further reference details] [Publication appraisal] [Free Full text]
● (12) Beynon R, Leeflang MM, McDonald S, Eisinga A, Mitchell RL, Whiting P, Glanville JM. Search strategies to identify diagnostic accuracy studies in MEDLINE and EMBASE. Cochrane Database of Systematic Reviews. 2013, Issue 9. Art. No.: MR000022. DOI: 10.1002/14651858.MR000022.pub3. [Further reference details] [Publication appraisal] [Free full text]