Open Peer Review with Authorea

The peer review process is a pillar of modern research, verifying and validating the ever-increasing output of academia. While the academic community agrees that some process of review is necessary to ensure the quality of published research, not everybody agrees on the best approach. In particular, doubts have been cast on the current peer review process: most journals select and assign one anonymous referee (few journals assign two or more) who is in charge of reviewing the manuscript and recommending it for publication or rejection. The argument is that the current peer review system is becoming inadequate. Here’s an incomplete list of issues:

  • Research is increasingly collaborative, complex, and specialized. Thus, it is less likely that one or a few referees can have the necessary expertise (and time) to properly handle many modern articles. Simply put, the average number of authors per paper has been steadily increasing in the last few decades, while the number of referees per paper has not.

  • “Publication pressure” means there is a growing number of papers to referee. This need can not be easily matched since scholars, who need to constantly publish and engage in the “funding race”, have less time to be dedicated to community service (in a “single referee” system the review process is very time consuming).

  • Given the anonymous nature of peer reviewing manuscripts, researchers who volunteer their valuable time and knowledge don’t get recognition for contributing.

  • Cases of peer-review scams, mostly from predatory open access publishers, have grown in number over recent years. A number of journals, exploiting the publication pressure climate, accept and publish articles with little or no peer review.

  • Similarly, there are reports of fraud in which authors review their own or close friends’ manuscripts to give favorable reviews (Ferguson 2014).

Number of authors per paper by decade in the field of medicine. Source: (Wren 2007). It’s interesting to note the average number of authors has increased steadily, but the number of referees per paper has not.

These failures seem to highlight two major requirements for a healthy peer review system.

  1. More scholars should participate in the peer review of an article. Exploiting the collective expertise of a research community would be the natural way to expedite and improve the quality of peer review.

  2. Peer review should be done in the open by researchers. This is the only way to ensure that no third parties can control or corrupt the process. The Wikipedia model shows this is a viable solution.

While the need to publish creates huge inertia for scholars to move away from the current paradigm, Authorea provides an excellent testbed to experiment with Open Peer Review. A first experiment of this kind is underway: a paper written on Authorea was submitted to a major journal and, on the same day, to the well known preprint repository arXiv. This is not news, and it’s done routinely by many scholars every day.

However, while the arXiv is a great tool for sharing research before it’s published or reviewed, users can’t comment or leave feedback on the content. That’s why the arXiv preprint was linked to its Authorea version, where readers can leave feedback using a powerful commenting system - and even link “discussion” or “response” documents to the original content.

So ultimately, the place where the paper was written becomes the place where the paper is discussed by the community. Since Authorea is also built for transparency and reproducibility – it allows you to include the data and code used during the research process – it is the ideal place for an effective review to take place.

This makes the review process more natural and seamless. The feedback left on Authorea becomes part of the paper itself. Doubts raised by an open reviewer can be answered by the authors and the wider community. Ideas or suggestions become commentaries that can trigger future collaborations and investigations.

At the end of the day, all this work is not lost and scatter. Credit can be acknowledged and distributed. And best of all, everything connected to the document (the text, data, code, figures, reviews, comments, future directions taken, etc.) becomes part of the legacy of the paper.