Background:
The presence of an a priori design, e.g. preregistration in PROSPERO or publication of a protocol, is an essential component of the systematic review [SR] process. Publication of a protocol confers the advantages of peer review. The time between protocol submission and acceptance [after revision, if necessary] is crucial for authors of SRs, because the SR cannot be started before completing the protocol.
Objectives:
To analyse the peer review process of published protocols and to explore trends over time.
Methods:
We searched PubMed to identify all protocols of SRs published in BMC Systematic Reviews up to December 2017. One author extracted data available from the protocol and the open access peer review into a standardised form, with a 10% sample checked by a second author. We judged the extent of revision [minor/major] on the basis of the reviewer comments. We investigated a further 10% sample regarding the reviewers' comments and authors' responses, using PRIMSA-P as a guideline.
Results:
We identified 545 published protocols; the number increased from 22 protocols in 2012 to 145 in 2017. Of the 485 protocols [89.0%] that were registered in PROSPERO, the majority [87.4%] were registered before submission of the protocol. There was no revision in 9.0%, one revision in 72.1%, two revisions in 17.3% and three or four revisions in 1.7% of published protocols. The median time from submission to first peer review increased over the years [2012: 29 days versus 2017: 52 days], as did the median time from submission to acceptance [2012: 70 days versus 2017: 142 days]. The proportion of protocols judged as being in need of major revision increased from 22.7% in 2012 to 52.4% in 2017.
Conclusions:
The number of major revisions increased in recent years, as did the duration of the peer review process. The time from submission to acceptance has increased to approximately five months, which significantly extend the time before a SR can be completed. As required by BMC Systematic Reviews, nearly all PROSPERO registrations took place before the submission of the protocol. Paradoxically, PROSPERO recommends registering when the protocol is complete, but this is just the case after the peer review is completed.
Patient or healthcare consumer involvement:
Due to the methodological character of this investigation, patient or healthcare consumer involvement was not planned.