Appraisal of: Relevo R, Paynter R. Peer Review of Search Strategies [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ) (US); 2012 Jun. Available from: http://www.ncbi.nlm.nih.gov/books/NBK98353/
This study was an evaluation of the feasibility of instituting a peer review process for search strategies in HTA organizations. The aims of the study were to:
1. evaluate whether the PRESS instrument (4) or no-instrument (‘free-form’ evaluations) was preferred by Technical Expert Peer Reviewers (TEPRs) of search strategies.
2. evaluate the usefulness of a peer review process for database search strategies, that is, did peer reviews change search strategies.
3. evaluate the costs of implementing a formal peer review of search strategies programme as a part of the review process.
The investigators undertook a randomized controlled trial of peer review using the PRESS instrument versus “free-form” review of search strategies, at the publication of the research protocol phase of AHRQ reports, by predominantly qualified and experienced information specialists. Of those participants randomized to the PRESS intervention (20/25) only 11 went on to complete the study and use the PRESS instrument. Of these, most took less than 2 hours to complete the review using PRESS (91%), some thought it could be incorporated into their workflow (yes: 46%; maybe 46%); some would have been willing to take on peer review duties on a more permanent basis (yes: 37%; maybe 37%); most found the instrument helpful (82%); 54% indicated that they preferred having the checklist but not being required to use it and 27% said that they preferred the PRESS instrument and being required to use it. The PRESS instrument reviews contained more recommendations overall and had more comments that could be termed error detection.