PubMed

Appraisal of: "Sampson M. Complementary approaches to searching MEDLINE may be sufficient for updating existing systematic reviews. J Clin Epidemiol 2016; 78: 108-115."

Short description: 

The aim of the article was to test whether the combined approach of a focused Boolean search paired with a second search using the similar articles feature of PubMed or support vector machine (SVM) can yield high recall with reasonable precision.

The general approach of a Boolean plus a ranking search is effective in MEDLINE retrieval for systematic reviews. Very high levels of identification of relevant MEDLINE records, with adequate precision, are possible using a focused Boolean search complemented by a document similarity or ranking method.

Limitations stated by the author(s): 

There are two limitations to our proposed strategy. Other databases should be searched in the unusual event that numerous studies, representing more than a small proportion of the total N, are not included in MEDLINE. Second, when it is important to find articles too new to be indexed by MEDLINE, systematic reviewers may wish to conduct a simple PubMed search limited to the nonindexed subsets.

Limitations stated by the reviewer(s): 
It is not possible to reproduce or use the support vector machine functionality.
Study Type: 
Single study

Appraisal of: "Boeker M., Vach W, Motschall E.. Time-dependent migration of citations through PubMed and OvidSP subsets: a study on a series of simultaneous PubMed and OvidSP searches. Stud Health Technol Inform. 2013;192:1196."

Reviewer(s): 
Short description: 

The aim of this study was to compare PubMed and OvidSP in terms of the timeliness of the different interfaces. The authors conclude OvidSP is updated within a period of two days.

Limitations stated by the author(s): 

No limits were discussed by the authors.

Limitations stated by the reviewer(s): 
This is a conference abstract so not enough detail is presented to ascertain the limitations.
Study Type: 
Single study

Appraisal of: Pohl, S. Boolean and Ranked Information Retrieval for Biomedical Systematic Reviewing [thesis]. Department of Computer Science and Software Engineering: University of Melbourne; 2012.

Short description: 

The aim of the study was to devise and examine ranked retrieval methods with regard to their applicability to the search tasks involved in the preparation and update of systematic reviews. A ranked retrieval alternative in the form of the p-norm extended Boolean retrieval (EBR) model was shown to provide advantages over the current Boolean approach, but to also satisfy important task specific requirements. Using this retrieval method, a greater number of relevant studies than with the current Boolean approach can be identified for inclusion in systematic reviews, reducing the dependence on potentially biased secondary search methods. The method is close to current practice and hence requires minimal changes to the current process, which should facilitate its adoption.

Limitations stated by the author(s): 

"One caveat with this experimental setup is that the queries could possibly be biased by knowledge about the documents in the test set. Hence, absolute performance on the test set might not be reliable, but this setup still allows for relative comparison of the approach because both the optimized queries as well as the baseline would have this advantage." (p. 154)

Limitations stated by the reviewer(s): 
No additional limitations were detected by the reviewers.
Study Type: 
Single study
Supplemental publications to the study: 

Appraisal of: Sampson M, Shojania KG, McGowan J, Daniel R, Rader T, Iansavichene AE et al. Surveillance search techniques identified the need to update systematic reviews. J Clin Epidemiol 2008; 61(8): 755-762.

Short description: 

The article reports on literature surveillance methods to identify new evidence eligible for updating systematic reviews. Five surveillance search approaches are tested in the context of identifying studies that would signal major or invalidating new evidence for existing systematic reviews of health care interventions.

Searches were tested in a cohort of 77 systematic reviews. No one method yielded consistently high recall of relevant new evidence, so combinations of the strategies were examined. A search algorithm based on PubMed’s related article search in combination with a subject searching using clinical queries was the most effective combination, retrieving all relevant new records in 68 cases. Citing RCT searching found unique material in only 2 cases (recall 37%).

Surveillance for emerging evidence that signals the need to update systematic reviews is feasible using a combination of subject searching and searching based on the PubMed’s related article function.

Limitations stated by the author(s): 

The retrospective observational design used here has certain limitations including absence of control over data collectiondin the case of the CENTRAL database, where indexing dates of records are not recorded; this limited our ability to determine when new evidence would have been available to reviewers. Our assessment of the point at which a review was in need of update could have been influenced by our access to subsequent confirmatory evidence, although any resulting bias would influence each search approach equally.

Limitations stated by the reviewer(s): 
No additional limitations were detected by the authors.
Study Type: 
Single study

Appraisal of: Mattioli S, Farioli A, Cooke RMT, Baldasseroni A, Ruotsalainen J, Placidi D, et al. Hidden effectiveness? Results of hand-searching Italian language journals for occupational health interventions. Occup Environ Med 2012;69(7):522-524.

Short description: 

The study compared the yield of hand-searching with optimised electronic search strategies in retrieving occupational health (OH) intervention studies published in a language other than English.

The authors systematically hand-searched and screened reports of OH intervention studies published in Italian in peer-reviewed scientific journals between 1990 and 2008. The Cochrane Occupational Safety and Health Review Groups (OSHRG) most sensitive search string retrieved all 16 articles published in the Italian language journals that were indexed in MEDLINE.

These findings suggest that a sensitive electronic search strategy may be able to find most of the OH interventions published in languages other than English that are indexed in MEDLINE.

Limitations stated by the author(s): 

No limitations were stated by the authors.

Limitations stated by the reviewer(s): 
The authors hand searched only 3 MEDLINE-indexed journals.
Study Type: 
Single study

Appraisal of: Sampson M. Updating searches for systematic reviews [dissertation]. Department of Information Studies: University of Aberystwyth; 2009.

Short description: 

This thesis examines methods for updating searches for systematic reviews of healthcare interventions on the base of 77 systematic reviews. The Related Article search showed the highest recall. Relative performance of the test searches was consistent regardless of whether the intervention was a drug, device or procedure. All searches showed variability across clinical areas, but Related Articles RCT showed the most consistency. The pairing of Related Article RCT and Clinical Query gave excellent recall of new relevant material.

Limitations stated by the author(s): 

Screening method, generalization of results on precision (cohort was only 10 AHRQ Evidence Reports), the selection of the sample may have implications for generalizability.

Limitations stated by the reviewer(s): 
No additional limitations were detected by the reviewers.
Study Type: 
Single study

Appraisal of: Linder SK, Kamath GR, Pratt GF, Saraykar SS, Volk RJ. Citation searches are more sensitive than keyword searches to identify studies using specific measurement instruments. J Clin Epidemiol. 2015; 68(4):412-417.

Short description: 

The aim of the study was to compare the effectiveness of two search methods in identifying studies that used the Control Preferences Scale (CPS), a health care decision-making instrument.

The literature was searched using two methods: (1) keyword searching using variations of ‘‘Control Preferences Scale’’ and (2) cited reference searching using two seminal CPS publications. Three bibliographic databases [PubMed, Scopus, and Web of Science (WOS)] and one full-text database (Google Scholar) were searched.

Keyword searches in bibliographic databases yielded high average precision (90%) but low average sensitivity (16%). PubMed was the most precise, followed closely by Scopus and WOS. The Google Scholar keyword search had low precision (54%) but provided the highest sensitivity (70%). Cited reference searches in all databases yielded moderate sensitivity (GS: 54%, WoS: 45%, Scopus 52%), but precision ranged from 35% to 75% with Scopus being the most precise.

When trying to identify many articles using a particular instrument, cited reference searching in Scopus or WOS is preferable to keyword searching in these resources or in PubMed/MEDLINE. However, in Google Scholar, a full-text database, a keyword search for CPS found more studies than the cited reference searches and precision was slightly higher.

If the goal is to identify a few studies that use a specific instrument, keyword searches of a bibliographic database may be the most efficient. When it is important to conduct a comprehensive search to maximize the number of studies found, both cited reference and keyword searches should be conducted in more than one database. 

Limitations stated by the author(s): 

“Our case study has limitations. The search results for the CPS may not be applicable to similar searches for other instruments. Other instruments may be more or less likely to be referred to in database records or article references. Results may also vary for a different information retrieval goal such as searching for a specific outcome reported in an instrument. In addition, our keyword search strategy was limited to the name of the instrument (and a variation of the named ‘‘preferences’’ instead of ‘‘preference’’) so that an identical search could be performed in each database. A more complex strategy that uses terms related to the instrument (ie, role preferences, patient involvement) could provide different results.”

Limitations stated by the reviewer(s): 
Authors did not state the recall in Google scholar for both articles used for citation tracking. It seems to be necessary to have initial article that introduced the instrument and the validation study.
Study Type: 
Single study

Appraisal of: O'Leary N, Tiernan E, Walsh D, Lucey N, Kirkova J, Davis MP. The pitfalls of a systematic MEDLINE review in palliative medicine: symptom assessment instruments. Am J Hosp Palliat Care. 2007; 24(3): 181-184.

Short description: 

The authors describe the usefulness of the related articles link in PubMed as part of a systematic MEDLINE review. Of 1181 citations identified, through a PubMed search, 10 articles met the indusion criteria. as did a further 21 articles identified through hand-searching the references of the 10 articles. The PubMed related articles link of the 10 articles yielded 15% of all reports finally included in the systematic review. The authors suggest that it is a useful tool in PubMed for revietrieving complex evidence.

Limitations stated by the author(s): 

No limitations stated by the authors.

Limitations stated by the reviewer(s): 
Different numbers are described in the result section: references that met the inclusion criteria and references finally included in the systematic review. This selection process is not clearly documented and it is not clear how many references finally included in the systematic review are identified through which source.
Study Type: 
Single study

Appraisal of: Katchamart, W., Faulkner A, Feldman B,Tomlinson G, Bombardier C.. PubMed had a higher sensitivity than Ovid-MEDLINE in the search for systematic reviews. J Clin Epidemiol 2011;64: 805-807

Reviewer(s): 
Short description: 

The aim of this study was to compare the performance of Ovid-MEDLINE vs. PubMed for identifying randomized controlled trials of methotrexate (MTX) in patients with rheumatoid arthritis (RA). They measured the performance of each strategies for Ovid-MEDLINE and PubMed for a systematic review of MTX in RA. They found that PubMed retrieved more citations overall than Ovid-MEDLINE. Of 20 citations that met eligibility criteria for the review, Ovid-MEDLINE retrieved 17 (85%) and PubMed 18 (90%). The precision was 0.881% (NNR: 114) for Ovid-MEDLINE vs. 0.884% (NNR:113)  for PubMed.

Limitations stated by the author(s): 

The study was based on the results of searches conducted for one systematic review of one specific topic. The generalisability of the results needs to be tested in other reviews.

Limitations stated by the reviewer(s): 
The authors did not include the Ovid ‘MEDLINE In-Process & Other Non-Indexed Citations’.
Study Type: 
Single study

Appraisal of: Irvin, E. Does it matter which version of MEDLINE you search?

Reviewer(s): 
Short description: 

Based on the conference abstract: The aim of the study is to determine if there is a difference in the yield of PubMed and OVID MEDLINE and to identify which has the highest yield. The researcher selected ten reviews by the Back, Palliative Care, and Supportive Care (PaPaS) Cochrane Review Group and then ascertained how many of the included studies were contained in PubMed and OVID Medline. In the 5 physical interventions reviews, PubMed contained 57% (43/76) and OVID MEDLINE 49% (37/76). The results of the other 5 reviews are not presented in the abstract. The author explain the higher number of references retrieved in PubMed was due to  the fact that PubMed indexes citations from journals that precede the date those same journals are selected for Ovid indexing. 

Limitations stated by the reviewer(s): 
This is a retrospective analysis, as such it is unable to take account of search strategy implementation and the potential for a search interface to affect the number of articles retrieved. They only checked if the references were indexed in the different interfaces, not the differences from carrying out actual search strategies.
Study Type: 
Single study
Syndicate content