The Editorial Board Members at Communications Biology are in the unique position of being authors, reviewers, and editors, so I asked a few of them to weigh in on the future of peer review.
Their responses have been edited for length and clarity. You will see that they agree on the need to reduce unconscious bias and to give reviewers credit for their time and effort. Still, each places emphasis on different aspects for improving peer review.
Editorial Board Members who contributed to this post:
Katie Davis, University of York, UK
Georgios Giamas, University of Sussex, UK
Quan-Xing Liu, East China Normal University, China
In your view, what have been some of the best innovations in peer review?
Katie Davis: Double blind reviews without a doubt. It is well known that unconscious bias affects the review process and double-blind peer review is such a good solution. I hope it will eventually be an option for all journals. Completely open reviews are another option here but I have concerns about confidentiality, particularly with regard to junior researchers feeling unable to be honest for fear of retaliation.
Georgios Giamas: I am not sure if they are any considerable innovations. However, I like the interactive review evaluation that some journals incorporate.
Quan-Xing Liu: Each reviewer has his own opinion and scope for the scientific standards, as has been outlined in Prof. Raymond Goldstein’s commentary on peer review in biological journals. In many cases, two referees will return contradictory reports for the same manuscript. This problem is being solved by some journals (e.g., eLife, Communications Biology) with novel peer reviewer processes. The essence of the eLife review process is an online discussion between the referees and the handling editor of a paper, so that they arrive at a single consensus report—a “decision letter”—that is sent to the authors, which avoids the problem of conflicting reports.
How would you like to see peer review evolve in the short- and long-term?
Katie Davis: I would like to see authors required to provide data in a usable, machine readable format. Not necessarily to be provided on publication, as there may be good reasons not to make all data available, but I think it is important for the reviewers to be able to check code and data for quality. It would also speed up the process as it causes significant delays when a reviewer has to request some of the data in order to be able to complete their review.
Another consideration is how we can integrate preprints, peer review and the final finished products into a more open access chain while maintaining confidentiality and quality. I would certainly like to see more journals encouraging preprints as these is particularly valuable for early career researchers.
Georgios Giamas: I would like to see shorter turnaround time. I also think that editors’ judgment/decision should not just rely on some reviewers’ comments. Instead, a more active role and higher responsibility for the editor is needed.
Quan-Xing Liu: Peer review is an indispensable process to scientific development that ensures the quality of the articles and accuracy of researched experiments since it was first employed in 1665 by the Royal Society’s Philosophical Transactions of The Royal Society. I have no idea how to change the peer review process in the short- and long-term in the future, but I would say that evaluating another researcher’s work hones critical thinking skills to both authors and reviewers. There is also the need for editors to make decisions that are not simply based on “counting votes” for rejection or revision. This is working well in our peer review process at Communications Biology. From my experiences, the more constructive feedback comes from experts from different academic backgrounds, and so I as an editor try to seek diverse reviewers to evaluate our manuscripts.
What role would you like authors, reviewers, and editors play in peer review moving forward?
Katie Davis: One thing I would like to see is authors required to provide reviewer suggestions (with certain guidelines/restrictions with regard to who may/may not be selected). They know their work the best and this could be really valuable. Editors would obviously still need to assign additional reviewers to balance any potential bias but I think this could improve the appropriateness of reviewers any given manuscript.
Georgios Giamas: Authors should respond to the point and not ‘diplomatically’ avoid certain requests if they are not able to perform them. Reviewers should reply faster and have realistic demands. Editors should take additional parameters into consideration when deciding about a manuscript. For example, the overall time spent on the whole review process, which translates to money for the specific lab. Also, editors need to keep in mind that simply requesting more data is not always advisable.
Quan-Xing Liu: Peer review is an indispensable process to scientific development that ensure the quality of the articles and accuracy of researched experiments. Of course, authors should be the champions of their own work. This also is true for myself. As an author, I try to get detailed feedback on both major and minor points from referees and editors because I want to improve my work and ultimately publish it. However, reviewers often seem to play the role of fault-finder, but being a reviewer is a powerful role, helping authors to improve their original manuscript to present a more complete picture, clearer figures, and a more widely accepted perception to reader. Editors act as a bridge between authors and reviewers, and academic editors are able to provide expert suggestions to the in-house editorial team.
How can we maintain high quality in peer review in new models of peer review?
Katie Davis: I think keeping some level of choice is important here. There may be good reasons for junior researchers to wish to maintain anonymity when reviewing. Enforcing complete open reviews could result in a decrease in uptake of review invitations, and it's already difficult to secure reviewers. For open reviews where reviewers' comments are public I think editors need to be careful not to publish any comments that are not constructive. Unfortunately I've seen plenty of comments (as an author and as an editor) that should not be made public.
Georgios Giamas: Editors should not send other manuscripts to reviewers that have already reviewed papers from the same group (whether they have accepted or rejected them), for a specific period of time (e.g. for ~2 years). Also, and this is more important for papers that are rejected, authors should have access to the comments that are currently being sent only to the editors.
Quan-Xing Liu: I would suggest that referees and editors work together to write the decision letter. Not all comments are equivalently important to help authors improve their manuscripts. Transparent peer review is another excellent way to push referees to provide their key criticisms and to help readers to see different perspectives, and which concerns and revisions are important in the published articles.
How would you like to see peer review rewarded?
Katie Davis: This is an interesting point, as it raises the issue of how we can reward peer review whilst still maintaining anonymity (if desired). Journals could record and publish the number of reviews each reviewer has contributed (though not the manuscript reviewed without explicit permission). It might still be necessary to think carefully about anonymity here in case individual reviewers can be identified. This is probably more of an issue if reviews are open as I feel that this could lead to a loss of anonymity. While completely open and transparent reviews seem like the ideal, I do think there's a risk that some reviewers - disproportionately junior academics - may feel unable to be completely honest in their reviews without the security of anonymity.
I think some kind of financial reward could be appropriate, not in the form of cash, but perhaps a small credit towards future publishing costs? Personally, I would find this to be a huge incentive, even if the credit is fairly small. Credits could even be transferable within publishers, e.g., a credit for reviewing a Communications Biology manuscript and a credit for reviewing a Nature Communications manuscript could be combined as two credits for any Nature Research group journal. In an attempt to speed up the review process, the reward might only be credited if the review is returned within the agreed-upon deadline(!).
Georgios Giamas: Some journals already have the ‘review of the month’ recognition. Since nowadays authors are receiving many requests for review, proper motivation should be given. Sadly or not, it should be a financial reward: for example, x amount for completing 3-5 reviews, y amount for >5…within a specific period of time.
Quan-Xing Liu: This is an innovative suggestion. But it would depend on the benefit being offered. I would suggest that peer reviewers providing high-quality reports could publish one of their manuscripts in Communications Biology (a model used by other journals, like PNAS). This could encourage referees to return thorough, constructive reports and to join reviewer teams.
How can we expand and improve the pool of available reviewers?
Katie Davis: Finding reviewers can be very challenging so I think this is a really important consideration. Databases of willing reviewers is one option. Prospective reviewers would ideally sign up to a central database that editors can access, though of course we are left with the question of who would pay for and maintain the database? The other way to do it would be for journals to maintain their own databases, which could potentially be a single database for each publisher to reduce the workload on both editors and reviewers. I have seen editors put calls out on Twitter for prospective reviewers but I am unsure how successful this approach is in practice; I certainly have never received review invitations as a result of responding to one of these calls but perhaps my name is now in the journal database for a future invitation.
Georgios Giamas: Better organise reviewer databases to include the average time that reviewers spent for the review process and the quality (score 1-5) of the review reports. Sister journals, or all the journals belonging to a publisher, should be able to share and have access to such databases and not reinvent the wheel.
Quan-Xing Liu: This largely depends on the reputation of the journal. Rewarding reviewers in some way could also be useful for attracting reviewers.