Diagnostic accuracy

Last revised: 


HTA may include assessment of new diagnostic technologies or techniques. These can involve the identification and review of diagnostic test accuracy (DTA) studies designed to differentiate between individuals with and without a target condition (1). The Cochrane Collaboration has published an evidence-based guide to searching for DTA studies, which provides the basis for this summary (2).

DTA studies tend to be poorly reported and searching for them can be problematic due to this inadequate reporting and inconsistent terminology, the absence of appropriate indexing terms in some databases for this publication type, and inconsistent use of suitable indexing terms where they are available (2).

Sources to search

Relying only on searching MEDLINE is not recommended, as it is unlikely to be the most comprehensive source of diagnostic information and because diagnostic studies are not easy to retrieve efficiently in bibliographic databases (3). Relative recall analysis of systematic reviews has also suggested other databases might yield additional studies including Science Citation Index, BIOSIS and LILACS (3). Recent analyses have suggested that fewer databases might be adequate, but are weakened by their reliance on known-item searches (4,5,6). Review searches may not detect all the records in MEDLINE that might be relevant to a review, so searching other databases provides opportunities to pick up (MEDLINE indexed) studies by other routes. An analysis of ten meta-analyses found that only using studies indexed in MEDLINE did not impact significantly on the sensitivity and specificity estimates of the meta-analyses in those reviews (4). A second analysis of 16 meta-analyses of diagnostic accuracy studies of depression screening tools found 94% (range: 83-100%) of the primary studies included in the meta-analyses were indexed in MEDLINE (5). The remaining non-MEDLINE indexed studies were located in Scopus, PsycINFO, and Embase. The authors acknowledged that the quality of the majority of the original reviews could not be determined. Another recent study of nine reviews performed by a single research group found that the reviewers’ original searches would have found 85% of their included studies from MEDLINE and Embase (range: 60-100%) (6). Adding reference checking to the process would have found 93% of the included studies. The Cochrane Handbook (2), based on available research-evidence, currently recommends that searches should include the following databases for reviews and primary studies:

●      MEDLINE
●      Embase
●      ARIF
●      HTA database
●      DARE (closed to new records from March 2015)
●      Cochrane Database of Systematic Reviews
●      Searches for unpublished studies in dissertations databases and grey literature databases
●      Reference checking
●      Citation searches in services such as Science Citation Index, Scopus, and Google Scholar, as well as related articles options in interfaces such as PubMed or Ovid.

In addition the following new databases could also be searched:

●     PROSPERO - register of systematic reviews
●     Epistemonikos - collection of systematic reviews and their included studies
●     PDQ Evidence - collection of systematic reviews about health systems and their included studies 

HTA agencies may also undertake assessments of diagnostic tests and so agency websites should also be explored, for example NICE diagnostic test guidance can be accessed here.

Although the proportion of ongoing studies investigating diagnostic test accuracy may still be relatively low (7), some are being recorded prospectively on trials registers such as ClinicalTrials.gov and the ICTRP portal (8).

The evidence for the value of handsearching is currently sparse, with one recent study of one topic showing that handsearching contributed little (9). It is possible that the topic of the research was well defined and the database searches were exemplary, and therefore the handsearching contribution would be different in other topics (9). More evidence is required on the yield and value of handsearching. Where a topic is published in journals that are not indexed in bibliographic databases, handsearching can still serve a purpose but this needs to be evaluated question by question.

Designing search strategies

Search strategies should be designed to be highly sensitive using a wide variety of search terms, both text words and subject indexing, to ensure that the many different ways that a test may be described feature in the search (2).  Information specialists should be aware of the weaknesses of reporting in abstracts of diagnostic accuracy studies. One exploratory study evaluating the comprehensiveness of reporting in the abstracts of 12 high-impact journals found 50% of the articles did not identify the study as a diagnostic accuracy study in the title and 65% included the sensitivity and/or specificity estimates in the abstract (10).

The search should reflect some, but not necessarily all, of the key concepts of the review (2). The search is likely to capture the index test being investigated and the target condition being diagnosed (2,10). A third set of terms can be considered to capture the patient description or the reference standard. The development of search strategies for DTA studies can be challenging and may involve several iterations to reach a strategy that captures the complex way records may present concepts of diagnosis (2). Cochrane Reviews of diagnostic test accuracy studies and the Cochrane handbook provide examples of search approaches for these, often complex, topics. Strategies may include both general terms (such as the generic type of diagnostic method, for example dipsticks) and specific terms such as named dipstick tests (2).

There are many published methodological search filters designed to capture studies of diagnostic test accuracy and including test measurement terms such as sensitivity and accuracy (11). The evidence, however, on the performance of DTA search filters suggests that combining filters with a search for a population and an index test is likely to miss relevant studies (12,13,14,15). Search filters for DTA studies do not seem to perform consistently and may result in unacceptable reductions in sensitivity (11,12,13,14,15,16). Some studies have found that there may be instances where these methodological filters could be used, but these are not within the context of information retrieval for the production of health technology assessments (17,18).  When all the research is considered together, current evidence suggests that for search strategies designed to support systematic reviews of diagnostic accuracy, as long as DTA filters are not the only approach, they may be useful as one component of a search strategy which involves several search approaches:  a “multi-stranded” approach involves multiple queries run sequentially and using different combinations of concepts. Search filters can be identified from the InterTASC Information Specialists' Sub-Group (ISSG) Search Filter Resource (11).

Subheadings (floating subheadings and subheadings attached to the index test or the target condition) may be a helpful component of the search strategy (2).

Developing a search strategy can be iterative and complex and it can be helpful to have topic experts to review samples of search results for relevance and it is always helpful to be able to test retrieval against sets of known relevant records.

When searching trials registers for records of ongoing DTA studies, a small study of thirty register entries found that 90% reported that the study was a DTA study in the title of the record, 97% reported the target condition and 100% reported the index test. This suggests that search terms to be used when searching registers can be relatively focused (8).

We have used the searching chapter of the Cochrane Handbook (2) as our baseline and SuRe Info appraisals have only been prepared for recently identified studies.


Reference list