The following questions and answers have been derived from the Cochrane Learning Live webinar session ‘Policy surgery’, delivered on May 15th 2018 by Bryony Urquhart (Editor, Cochrane Editorial and Methods Department) and Anupa Shah (Managing Editor Support of the Cochrane Editorial and Methods Department, Managing Editor of Cochrane Eyes and Vision). The questions and answers are grouped under the following headings:
Q. Will the changes to the CRGs peer review policy also affect the DTA peer review policy?
A. DTA peer review is subject to the same policy. As with CRGs, the DTA editorial team has until Jan 2019 to implement the policy fully. The DTA ET will be responsible for all elements of the policy, and providing the necessary information to the CRG. How they do that has yet to be decided. I will be talking to the DTA team (as with all CRGs) to ensure that they have the support they need to comply fully with the policy.
Q. What is the definition of a “clinical/topic specialist" in relation to the peer review policy guidance for implementation section.
A. The definition of clinical/topic specialist is quite broad as the intention is to describe anyone who has in-depth knowledge in the topic area covered by the review. We assume in most cases that this will be a clinician of some sort but may not always be. At protocol stage this person should be able to assess whether the research question is valid, and at review stage whether the research question is still valid (if not, what’s changed?), identify whether any relevant and important studies have been excluded, and whether the clinical context is correct and up-to-date.
The policy will be updated to reflect this definition.
A. Yes, until Jan 2019 use your own judgement. Any reason is valid as long as it is recorded. If you are concerned that most of your peer reviewers, or (as in this case) a certain category of peer reviewer, are opting to remain anonymous let us know. Any deviations from policy from January 2019 onwards will need to be sanctioned by David Tovey and exceptions to the policy listed in the EPPR and on the relevant group website.
We will be talking to each CRG individually to assess whether this is a particular concern and to put in place steps to monitor opt-outs prior to the implementation deadline.
Q. Are peer reviewers just external people or could they also be for example the Managing Editor who checks the MECIR checklists and has comments based on the list?
A. Anyone who makes comments during the peer review process (including anyone adding to, or prioritising peer reviewer comments – e.g. an Editor) needs to be named with their comments when submitted to the authors. A Managing Editor checking the MECIR checklist should be viewed as part of the editorial process.
Q. At present peer reviewers have the option of anonymity and you will collect reasons why people are opting out. In terms of decision point, should the deadline for collecting opt out reasons be until December and then you have one month to make the decision for implementing the policy?
A. We will work with groups to monitor this during the implementation phase, and the decision point will be December 2018. Cochrane does want to move towards a named peer review policy by January 2019 but at the same time we do not want to harm CRGs where the switch to named peer review will make it difficult for these CRGs to recruit peer reviewers (or certain types of peer reviewers).
Q. Should peer reviewers be able to opt out of being listed on the website?
A. At present, this is something we will continue to offer the peer reviewer (as we have done historically) but peer reviewers will still be able to opt out of being listed on the website after January 2019. The main thing is to ensure the named peer review process between author and peer reviewer occurs to ensure transparency in the decision-making process.
Q. What is the process for collecting feedback from CRGs about implementation of peer review? Will you be contacting Groups or should Groups contact you?
A: Bryony will contact Groups. As this will be over a period of time, the intention is to contact Groups initially but around Dec 2018/Jan 2019 it may be that we talk about implementation in terms of Networks.
Q. In our Group, we have given the option of being anonymous for years with an uptake of the option of 5% to 10%. In general, reasons given as junior staff that don’t want to be seen as too tough on senior people who might be evaluating them in the future or friends of the authors who do not want to challenge the relationship with the authors by providing candid comments.
A. This is a challenge but one that is within the ethos of Cochrane. By making the decision-making process between author and peer reviewer transparent, and making it mandatory, we hope to effect a shift in that perception within that community.
Q. In the peer review policy, it states that peer reviewers should always be offered the option of acknowledgement in the review. In the guidance, it states that peer reviewers should be acknowledged. Does that mean that it is now mandatory for Groups to acknowledge all peer reviewers in a review?
A. It is mandatory to ask but not mandatory to do it as the peer reviewer still has the option to opt out of being acknowledged. Groups should also obtain agreement of authors to allow the peer reviewer to be acknowledged as the authors have ownership of the protocol or review.
Q. In a review there may be peer reviewers who agree to be acknowledged and those who opt out and hence you get an incomplete listing. A reader could get the wrong impression that there has only been 1 peer reviewer when in fact there have been 4. What do you do in this situation?
A. It would be appropriate to include a statement such as “and we also would like to acknowledge all peer reviewers who wish to remain anonymous” or if you want to be specific you could state how many and thus replace all with the number.
Q. How do we deal with peer reviewers who do not have affiliations such as consumers?
A. The stipulation for included affiliations was to avoid doubt where there are similar names and there could be a case of mistaken identity. It’s appropriate to include names without affiliations where there are no affiliations to include.
Q. Does the peer review form now have the correct wording to ask the peer reviewers if they want to preserve anonymity and be acknowledged on the website and review?
A. Yes that text is now in the peer review forms that you can download from the EPPR. In addition, the peer reviewers are asked how they wish their name and acknowledgement to appear, so this should make it easier to attribute the comments when collating and sending to the author.
Conflict of interest
Q. When do the author COI statements become ‘out of date’? Specifically, do they need to be redone or re-evaluated between submission and publication of the review?
A. We are currently reviewing as there is no set policy/guidance for this. At the present time, if COI has been signed/updated at review submission stage this can be considered valid for at least 18 months.
Authorship and updates
Q. Do we have a policy statement that review authors must commit to do an update (e.g. in two years, sooner if data have emerged, or later if not)? And, if authors do not comply (e.g. stalling or not being responsive), are we (the editors of the review group) absolutely entitled to look for new authors who can commit to complete the review update on time? In which case, can we remove the old authors from the update?
A. Current licence for publication form states that authors agree to update the review in-line with current Cochrane policies. The current Cochrane policy states that a Cochrane Review should be updated based upon need. Having said that, the authorship of updates is a policy that we’re currently working on. We are considering when original authors can be decommissioned from an update and under what circumstances, and what rights they have in the updated review and how they should be acknowledged.
Q. The MOU signed between Central Cochrane and CRGs stipulates “1.2.6: Reject a review at any stage if judged not to be a priority: if found to overlap with another Cochrane Review; if concerns are raised about conflicts of interest or other aspects of publication ethics; or if the review teams cannot demonstrate sufficient capacity, expertise, or resources (including time) to carry out the review.” The policy manual and the draft rejection emails do not support this and instead focuses on timeliness, quality/competence and ethics.
Are we allowed to reject a review based on the topic and/or the ability of that review to answer the questions posed in the protocol? If so, then it would be helpful to reflect that in the examples provided in the policy. Or should we be taking another path to reject this draft protocol?
A. A gap has been identified here and we are currently working towards identifying wording to address this, and the policy will be updated accordingly.
Sometimes a protocol is required as decisions cannot be taken on title alone. Think you are definitely within policy to reject a protocol on the basis of topic/competence of author team or priority. It is less clear if the outright rejection of a review near completion on the basis of priority alone is appropriate.
Although the aim is to focus on rejecting more reviews to focus on publishing fewer higher quality reviews and preserving resources at CRG, peer review, etc., rejecting a review near completion would be an extreme situation and the authors would most likely appeal against the decision.
Q. What if the review was a priority question in 2016 but not a priority review in 2018?
A. One option would be to write to the authors giving them a deadline for completion and explaining that the review is no longer a priority and stating the reason. This statement could also be added to the UCS at publication so that readers are aware that the review will not be updated because it is no longer a priority. If the authors do not meet the deadline set by you, you are then able to apply the rejection policy, and the authors will be free to publish the review elsewhere.