Randomized controlled trials

Clinical trials that involve at least one test treatment and one control treatment, concurrent enrollment and follow-up of the test- and control-treated groups, and in which the treatments to be administered are selected by a random process, such as the use of a random-numbers table.

Appraisal of: Farrah K, Raab D. Errata for trial publications are not uncommon, are frequently not trivial, and can be challenging to access: a retrospective review. J Med Libr Assoc 2019; 107(2):187-193.

Reviewer(s): 
Short description: 

This paper conducted a retrospective review across forty systematic reviews published by CADTH in 2015, searching for errata associated with these reviews via PubMed, biomedical databases and journal publishers. The authors had three objectives: to estimate frequency of errata, evaluate impact of errata on the reviews, and explore how easily errata can be found through standard systematic review search methods.

The study found searching for errata was not straightforward as no single database provided all errata, the significance of time lag between publication of an original article and subsequent errata (overall median 237 days but a range of 15 to 1036 days), and how errata were indexed and linked to original articles. The authors rated errata identified as major, minor or trivial in terms of impact on study outcomes and found 16% of errors important enough to potentially affect study conclusions, overall half of all errors could impact on interpretation of study data.

The authors recommend checking appropriate settings in bibliographic software to make sure errata can easily be identified, and when updating systematic reviews searchers should consult the Cochrane handbook for current best practice to identify errata.

Limitations stated by the author(s): 

This paper was based on a small sample of errata and looked only at systematic reviews of single drugs.

The authors did not look at the impact of these errata on the conclusions of the systematic reviews.

Limitations stated by the reviewer(s): 
No specific sub-analysis of the issues or when these studies were originally published was undertaken. Not clear what actions were taken (or how and when they should subsequently be undertaken or recorded) to rectify errata in currently published systematic reviews.
Study Type: 
Review

Appraisal of: Chapman AL, Morgan LC, Gartlehner G. Semi-automating the manual literature search for systematic reviews increases efficiency. Health Info Libr J. 2009; 27(1):22-7.

Short description: 

Manual literature searching (or hand-searching) for systematic reviews is often promoted as a way of minimizing retrieval bias (the failure to find relevant studies due to the retrieval capacity of electronic databases), but it can be a lengthy and unsystematic process. This study aimed to determine the validity of a new manual search method to check reference lists of identified studies for relevant articles using the Scopus database by comparing it to the traditional method of manually checking reference lists as the gold standard. Outcome measures included accuracy and completeness of article detection (validity) and personnel time involved (efficiency). The Scopus method identified the same studies as the gold standard, but was much more time efficient (3 hours versus 8 hours, time saving of 62.5%). The study authors concluded that the Scopus method could significantly improve the efficiency of manual searches and thus of systematic reviews.

Limitations stated by the author(s): 

Findings are based on one medium-sized systematic review. Further method studies are needed to validate these findings and identify potential improvement to this approach.

 Scopus requires a paid subscription.

Scopus currently does not include the reference lists of Cochrane reviews, which is a major limitation as Cochrane reviews are a major source of trial information.

Limitations stated by the reviewer(s): 
Findings based on reference checking of only 20 studies.
Study Type: 
Single study

Appraisal of: Golder S, Loke YK, Bland M. Meta-analyses of adverse effects data derived from randomised controlled trials as compared to observational studies: methodological overview. PLoS Med 2011;8(5):e1001026.

Reviewer(s): 
Short description: 
This methodological overview investigates the impact of study design in meta-analyses of adverse effects data. The authors look at the estimates of the risk of adverse events derived from meta-analyses of randomized controlled trials (RCTs) and from observational studies. The evidence indicates that there is no difference on average between estimates of harm in meta-analyses of RCTs as compared to observational studies. The study suggests that systematic reviews of harms should not be restricted to specific study types.
Limitations stated by the author(s): 
Confounding factors may account for any differences identified when comparing pooled results from various study designs. Due to time constraints, data was not extracted from individual papers; rather the data reported by the authors of the meta-analyses were used. Because the search strategy was designed to identify methodological papers whose primary goal was to assess the influence of study design on adverse effects, systematic reviews with embedded data comparing study designs may have been missed. The considerable heterogeneity between the comparisons of different studies suggests that any differences could be specific to particular types of interventions or certain adverse events. For example, some types of adverse events may be more easily identified by RCTs as opposed to observational studies.
Limitations stated by the reviewer(s): 
No additional limitations detected by the reviewer
Study Type: 
Review
Related Chapters: 

Comments from the authors:

Syndicate content