PubMed
Appraisal of: Farrah K, Raab D. Errata for trial publications are not uncommon, are frequently not trivial, and can be challenging to access: a retrospective review. J Med Libr Assoc 2019; 107(2):187-193.
This paper conducted a retrospective review across forty systematic reviews published by CADTH in 2015, searching for errata associated with these reviews via PubMed, biomedical databases and journal publishers. The authors had three objectives: to estimate frequency of errata, evaluate impact of errata on the reviews, and explore how easily errata can be found through standard systematic review search methods.
The study found searching for errata was not straightforward as no single database provided all errata, the significance of time lag between publication of an original article and subsequent errata (overall median 237 days but a range of 15 to 1036 days), and how errata were indexed and linked to original articles. The authors rated errata identified as major, minor or trivial in terms of impact on study outcomes and found 16% of errors important enough to potentially affect study conclusions, overall half of all errors could impact on interpretation of study data.
The authors recommend checking appropriate settings in bibliographic software to make sure errata can easily be identified, and when updating systematic reviews searchers should consult the Cochrane handbook for current best practice to identify errata.
Appraisal of: "Sampson M. Complementary approaches to searching MEDLINE may be sufficient for updating existing systematic reviews. J Clin Epidemiol 2016; 78: 108-115."
The aim of the article was to test whether the combined approach of a focused Boolean search paired with a second search using the similar articles feature of PubMed or support vector machine (SVM) can yield high recall with reasonable precision.
The general approach of a Boolean plus a ranking search is effective in MEDLINE retrieval for systematic reviews. Very high levels of identification of relevant MEDLINE records, with adequate precision, are possible using a focused Boolean search complemented by a document similarity or ranking method.
Appraisal of: "Boeker M., Vach W, Motschall E.. Time-dependent migration of citations through PubMed and OvidSP subsets: a study on a series of simultaneous PubMed and OvidSP searches. Stud Health Technol Inform. 2013;192:1196."
The aim of this study was to compare PubMed and OvidSP in terms of the timeliness of the different interfaces. The authors conclude OvidSP is updated within a period of two days.
Appraisal of: Pohl, S. Boolean and Ranked Information Retrieval for Biomedical Systematic Reviewing [thesis]. Department of Computer Science and Software Engineering: University of Melbourne; 2012.
The aim of the study was to devise and examine ranked retrieval methods with regard to their applicability to the search tasks involved in the preparation and update of systematic reviews. A ranked retrieval alternative in the form of the p-norm extended Boolean retrieval (EBR) model was shown to provide advantages over the current Boolean approach, but to also satisfy important task specific requirements. Using this retrieval method, a greater number of relevant studies than with the current Boolean approach can be identified for inclusion in systematic reviews, reducing the dependence on potentially biased secondary search methods. The method is close to current practice and hence requires minimal changes to the current process, which should facilitate its adoption.
Appraisal of: Sampson M, Shojania KG, McGowan J, Daniel R, Rader T, Iansavichene AE et al. Surveillance search techniques identified the need to update systematic reviews. J Clin Epidemiol 2008; 61(8): 755-762.
The article reports on literature surveillance methods to identify new evidence eligible for updating systematic reviews. Five surveillance search approaches are tested in the context of identifying studies that would signal major or invalidating new evidence for existing systematic reviews of health care interventions.
Searches were tested in a cohort of 77 systematic reviews. No one method yielded consistently high recall of relevant new evidence, so combinations of the strategies were examined. A search algorithm based on PubMed’s related article search in combination with a subject searching using clinical queries was the most effective combination, retrieving all relevant new records in 68 cases. Citing RCT searching found unique material in only 2 cases (recall 37%).
Surveillance for emerging evidence that signals the need to update systematic reviews is feasible using a combination of subject searching and searching based on the PubMed’s related article function.
Appraisal of: Mattioli S, Farioli A, Cooke RMT, Baldasseroni A, Ruotsalainen J, Placidi D, et al. Hidden effectiveness? Results of hand-searching Italian language journals for occupational health interventions. Occup Environ Med 2012;69(7):522-524.
The study compared the yield of hand-searching with optimised electronic search strategies in retrieving occupational health (OH) intervention studies published in a language other than English.
The authors systematically hand-searched and screened reports of OH intervention studies published in Italian in peer-reviewed scientific journals between 1990 and 2008. The Cochrane Occupational Safety and Health Review Groups (OSHRG) most sensitive search string retrieved all 16 articles published in the Italian language journals that were indexed in MEDLINE.
These findings suggest that a sensitive electronic search strategy may be able to find most of the OH interventions published in languages other than English that are indexed in MEDLINE.
Appraisal of: Sampson M. Updating searches for systematic reviews [dissertation]. Department of Information Studies: University of Aberystwyth; 2009.
This thesis examines methods for updating searches for systematic reviews of healthcare interventions on the base of 77 systematic reviews. The Related Article search showed the highest recall. Relative performance of the test searches was consistent regardless of whether the intervention was a drug, device or procedure. All searches showed variability across clinical areas, but Related Articles RCT showed the most consistency. The pairing of Related Article RCT and Clinical Query gave excellent recall of new relevant material.
Appraisal of: Linder SK, Kamath GR, Pratt GF, Saraykar SS, Volk RJ. Citation searches are more sensitive than keyword searches to identify studies using specific measurement instruments. J Clin Epidemiol. 2015; 68(4):412-417.
The aim of the study was to compare the effectiveness of two search methods in identifying studies that used the Control Preferences Scale (CPS), a health care decision-making instrument.
The literature was searched using two methods: (1) keyword searching using variations of ‘‘Control Preferences Scale’’ and (2) cited reference searching using two seminal CPS publications. Three bibliographic databases [PubMed, Scopus, and Web of Science (WOS)] and one full-text database (Google Scholar) were searched.
Keyword searches in bibliographic databases yielded high average precision (90%) but low average sensitivity (16%). PubMed was the most precise, followed closely by Scopus and WOS. The Google Scholar keyword search had low precision (54%) but provided the highest sensitivity (70%). Cited reference searches in all databases yielded moderate sensitivity (GS: 54%, WoS: 45%, Scopus 52%), but precision ranged from 35% to 75% with Scopus being the most precise.
When trying to identify many articles using a particular instrument, cited reference searching in Scopus or WOS is preferable to keyword searching in these resources or in PubMed/MEDLINE. However, in Google Scholar, a full-text database, a keyword search for CPS found more studies than the cited reference searches and precision was slightly higher.
If the goal is to identify a few studies that use a specific instrument, keyword searches of a bibliographic database may be the most efficient. When it is important to conduct a comprehensive search to maximize the number of studies found, both cited reference and keyword searches should be conducted in more than one database.
Appraisal of: O'Leary N, Tiernan E, Walsh D, Lucey N, Kirkova J, Davis MP. The pitfalls of a systematic MEDLINE review in palliative medicine: symptom assessment instruments. Am J Hosp Palliat Care. 2007; 24(3): 181-184.
The authors describe the usefulness of the related articles link in PubMed as part of a systematic MEDLINE review. Of 1181 citations identified, through a PubMed search, 10 articles met the indusion criteria. as did a further 21 articles identified through hand-searching the references of the 10 articles. The PubMed related articles link of the 10 articles yielded 15% of all reports finally included in the systematic review. The authors suggest that it is a useful tool in PubMed for revietrieving complex evidence.
Appraisal of: Katchamart, W., Faulkner A, Feldman B,Tomlinson G, Bombardier C.. PubMed had a higher sensitivity than Ovid-MEDLINE in the search for systematic reviews. J Clin Epidemiol 2011;64: 805-807
The aim of this study was to compare the performance of Ovid-MEDLINE vs. PubMed for identifying randomized controlled trials of methotrexate (MTX) in patients with rheumatoid arthritis (RA). They measured the performance of each strategies for Ovid-MEDLINE and PubMed for a systematic review of MTX in RA. They found that PubMed retrieved more citations overall than Ovid-MEDLINE. Of 20 citations that met eligibility criteria for the review, Ovid-MEDLINE retrieved 17 (85%) and PubMed 18 (90%). The precision was 0.881% (NNR: 114) for Ovid-MEDLINE vs. 0.884% (NNR:113) for PubMed.
