Scopus

Appraisal of: Sampson M, Shojania KG, McGowan J, Daniel R, Rader T, Iansavichene AE et al. Surveillance search techniques identified the need to update systematic reviews. J Clin Epidemiol 2008; 61(8): 755-762.

Short description: 

The article reports on literature surveillance methods to identify new evidence eligible for updating systematic reviews. Five surveillance search approaches are tested in the context of identifying studies that would signal major or invalidating new evidence for existing systematic reviews of health care interventions.

Searches were tested in a cohort of 77 systematic reviews. No one method yielded consistently high recall of relevant new evidence, so combinations of the strategies were examined. A search algorithm based on PubMed’s related article search in combination with a subject searching using clinical queries was the most effective combination, retrieving all relevant new records in 68 cases. Citing RCT searching found unique material in only 2 cases (recall 37%).

Surveillance for emerging evidence that signals the need to update systematic reviews is feasible using a combination of subject searching and searching based on the PubMed’s related article function.

Limitations stated by the author(s): 

The retrospective observational design used here has certain limitations including absence of control over data collectiondin the case of the CENTRAL database, where indexing dates of records are not recorded; this limited our ability to determine when new evidence would have been available to reviewers. Our assessment of the point at which a review was in need of update could have been influenced by our access to subsequent confirmatory evidence, although any resulting bias would influence each search approach equally.

Limitations stated by the reviewer(s): 
No additional limitations were detected by the authors.
Study Type: 
Single study

Appraisal of: Wright K, Golder S, Rodriguez-Lopez R. Citation searching: a systematic review case study of multiple risk behaviour interventions. BMC Med Res Methodol. 2014; 14:73.

Short description: 

Study about the effectiveness of using the citation sources Google Scholar, Scopus, Web of Science and OVIDSP MEDLINE to identify records for inclusion in a systematic review about multiple risk behaviour interventions.

40 included studies were used identified by traditional database searches from one systematic review. Each of the included studies was searched for in the four citation sources to retrieve the details of all papers that have cited these studies.

The highest number of citations was retrieved from Google Scholar (1680), followed by Scopus (1173), then Web of Science (1095) and lastly OVIDSP (213). To retrieve all the records identified by the citation tracking searching all four resources was required. Google Scholar identified the highest number of unique citations. The citation tracking identified 9 studies that met the review’s inclusion criteria (recall 22,5%).

Conclusion: Citation searching as a supplementary search method for systematic reviews may not be the best use of valuable time and resources.

Limitations stated by the author(s): 

In terms of searching, the scoping review had very broad coverage. Its aim was to identify any intervention promoting change in at least two risk behaviours and the search strategy incorporated terms for all of these (smoking, alcohol misuse, physical inactivity, unhealthy diet, illicit drug use, sexual risk behaviour, lack of seat belt use, lack of motorcycle/bicycle helmet use, lack of sunscreen use, gambling, poor oral hygiene and drink driving) in various set combinations. The resulting complexity will almost certainly have had an impact upon the overall performance of the database search strategies. As with any case study, there is uncertainty about how far the results of this study can be generalised, especially to other reviews with a more restrictive focus.

Limitations stated by the reviewer(s): 
No additional limitations were detected by the reviewers.
Study Type: 
Single study

Appraisal of: Chapman AL, Morgan LC, Gartlehner G. Semi-automating the manual literature search for systematic reviews increases efficiency. Health Info Libr J. 2009; 27(1):22-7.

Short description: 

Manual literature searching (or hand-searching) for systematic reviews is often promoted as a way of minimizing retrieval bias (the failure to find relevant studies due to the retrieval capacity of electronic databases), but it can be a lengthy and unsystematic process. This study aimed to determine the validity of a new manual search method to check reference lists of identified studies for relevant articles using the Scopus database by comparing it to the traditional method of manually checking reference lists as the gold standard. Outcome measures included accuracy and completeness of article detection (validity) and personnel time involved (efficiency). The Scopus method identified the same studies as the gold standard, but was much more time efficient (3 hours versus 8 hours, time saving of 62.5%). The study authors concluded that the Scopus method could significantly improve the efficiency of manual searches and thus of systematic reviews.

Limitations stated by the author(s): 

Findings are based on one medium-sized systematic review. Further method studies are needed to validate these findings and identify potential improvement to this approach.

 Scopus requires a paid subscription.

Scopus currently does not include the reference lists of Cochrane reviews, which is a major limitation as Cochrane reviews are a major source of trial information.

Limitations stated by the reviewer(s): 
Findings based on reference checking of only 20 studies.
Study Type: 
Single study

Appraisal of: Linder SK, Kamath GR, Pratt GF, Saraykar SS, Volk RJ. Citation searches are more sensitive than keyword searches to identify studies using specific measurement instruments. J Clin Epidemiol. 2015; 68(4):412-417.

Short description: 

The aim of the study was to compare the effectiveness of two search methods in identifying studies that used the Control Preferences Scale (CPS), a health care decision-making instrument.

The literature was searched using two methods: (1) keyword searching using variations of ‘‘Control Preferences Scale’’ and (2) cited reference searching using two seminal CPS publications. Three bibliographic databases [PubMed, Scopus, and Web of Science (WOS)] and one full-text database (Google Scholar) were searched.

Keyword searches in bibliographic databases yielded high average precision (90%) but low average sensitivity (16%). PubMed was the most precise, followed closely by Scopus and WOS. The Google Scholar keyword search had low precision (54%) but provided the highest sensitivity (70%). Cited reference searches in all databases yielded moderate sensitivity (GS: 54%, WoS: 45%, Scopus 52%), but precision ranged from 35% to 75% with Scopus being the most precise.

When trying to identify many articles using a particular instrument, cited reference searching in Scopus or WOS is preferable to keyword searching in these resources or in PubMed/MEDLINE. However, in Google Scholar, a full-text database, a keyword search for CPS found more studies than the cited reference searches and precision was slightly higher.

If the goal is to identify a few studies that use a specific instrument, keyword searches of a bibliographic database may be the most efficient. When it is important to conduct a comprehensive search to maximize the number of studies found, both cited reference and keyword searches should be conducted in more than one database. 

Limitations stated by the author(s): 

“Our case study has limitations. The search results for the CPS may not be applicable to similar searches for other instruments. Other instruments may be more or less likely to be referred to in database records or article references. Results may also vary for a different information retrieval goal such as searching for a specific outcome reported in an instrument. In addition, our keyword search strategy was limited to the name of the instrument (and a variation of the named ‘‘preferences’’ instead of ‘‘preference’’) so that an identical search could be performed in each database. A more complex strategy that uses terms related to the instrument (ie, role preferences, patient involvement) could provide different results.”

Limitations stated by the reviewer(s): 
Authors did not state the recall in Google scholar for both articles used for citation tracking. It seems to be necessary to have initial article that introduced the instrument and the validation study.
Study Type: 
Single study
Syndicate content