Written by Lindsay Morton

Three years ago PLOS implemented our version of open peer review, which we named with our typical scrupulous precision and total disregard for word count: Published Peer Review History. Since then, the PLOS journals have together published over 18,000 articles with accompanying peer review history. Were pleased to share more preliminary observations based on our first three years of data below. But first, a bit of background

Open peer review can mean many different things. The main unifying feature is an element of openness somewhere in the peer review process, which can range from public comments on all submissions, to an open decision letter after acceptance. PLOS version of open peer review is simple, flexible and modular, dictated by the choices reviewers and authors make during the peer review and publication process.

All PLOS journals default to anonymous peer review. Reviewers have the option to sign their peer reviews if they wish to do so, and are advised of the potential for the review to be published. After an article has been accepted for publication, authors can opt-in to publish the Peer Review History, which includes the key documents from the peer review assessment:

The result is four possible gradations of openness in peer review.

It is very important, we think, to note that the benefits of Published Peer Review History are by no means predicated on reviewers signing their reviews. While reviewers have the opportunity to claim credit if they so choose, the value to the wider scholarly community stems from the availability of their comments. We want to make this absolutely clear because, too often, critique of open peer review can focus on signing, missing the more vital aspect of constructive feedback.

PLOS Peer Review History is organized, time stamped, and machine readable, with the goal of enabling future meta-research. Each element of the history package has a sub-DOI and is tagged and indexed, making it easy to discover and cite, even if youre not reading it on one of the PLOS journal websites. Its also included in AllofPLOS for easier mining and analysis.

You might be thinking so what? When PLOS introduced Peer Review History in 2019, various versions of public review had been available in the scholarly marketplace for a decade or moreat BMJ, EMBO Press, and eLife, just to name a few.

All implementations of open reviewsigned and unsigned, posted as public commentary or published after the factserve to improve transparency and accountability, to demonstrate the quality of peer review and reinforce the validity of the published research, to acknowledge and honor peer reviewers contributions to the literature, and to enrich the scientific record with more expertise.

Our own particular version emphasizes flexibility and author control. After acceptance, PLOS authors must click a check box and opt-in to have their peer reviews published. As a large international publisher with a very broad scope, our goal with Published Peer Review History, as with Open Science in general, has been to bring openness within easy reach for all, while at the same time respecting researchers different values and requirements across disciplines, career stages, and regions. Opt-in works at scale for all the researchers across our portfolio.

The opt-in model is somewhat unusual, as compared to the more established pioneers in Open Peer Review, many of whom mandate openness (e.g. BMJ) or use an opt-out model (e.g. EMBO Press). PLOS opt-in approach results in lower uptake: on average, 40% of PLOS authors opt-in, as compared to the reported greater than 95% of EMBO Press authors who dont opt-out. At the same time, PLOS large publication volume means that, even with a lower rate of uptake, the journals contribute significantly to the total number of articles with public peer reviews available across the published literature. (PLOS journals together published nearly 18,000 articles in 2020. For comparison, the same year eLife published 1,870 articles, BMJ Open published 3,610*, and the five EMBO Press journals published 1,063*.) This absolute scale is helpful with normalizing practices and behaviors.

The opt-in model, in combination with our size and breadth of scope also produces a wealth of data, which we hope can help us to better understand the communities we serve, and inform our future direction with regard to published review.

Since PLOS Published Peer Review History was first introduced, opt-in has remained a remarkably consistent 40%, year after year.**

PLOS two highly selective journals, PLOS Biology and PLOS Medicine, also saw the highest rates of Published Peer Review opt-in among accepted authors, followed by two of the four community journals, PLOS Computational Biology, and PLOS Genetics. PLOS ONE and the remaining two community journals, PLOS Pathogens and PLOS NTDs saw the lowest opt-in rates. PLOS ONEs broad scope and high publication volume accounts for its lower average opt-in rate. Lower adoption among Pathogens and NTDs authors as compared with the other community journals may likewise relate to subject-specific norms and preferences.

When PLOS ONE opt-in rates are broken out by discipline a pattern of subject-area preference becomes apparent. In fields with more than 100 published articles in the time period, opt-ins were strongest in computational biology, public health fields (womens health, public health, mental health), research on caregiving (healthcare, nursing), and specific medical disciplines (emergency medicine, neurology, obstetrics, urology, anesthesiology, pulmonology, clinical trials, etc.). Lower opt-in rates were observed in engineering, math, chemistry, materials science, earth science, geography and various branches of biology (molecular biology, microbiology, cell biology, biotechnology), as well as in fields where medical and bench science intersect, such as basic cancer research.

While clinical in their potential translational applications, PLOS NTDs and PLOS Pathogens frequently publish research on the molecular and cellular aspects of disease, and therefore might expect commensurately lower opt-in rates.

However, PLOS Biology also publishes in the biological fields where PLOS ONE opt-in tends to be weakeryet it has the highest opt-in rate of any journal. Why is that? Perhaps, the journals hands-on, personalized editorial and peer review process influences author opt-in. This would be a fascinating area for investigation.

PLOS has always allowed reviewers to sign their reviews if they choose, but it wasnt until the system updates introduced at the same time as Published Peer Review History (May 22, 2019) that we gained the ability to easily and systematically measure signing. Since then, weve received 49,555 signed reviews (17.81% of all completed reviews). Those signed reviews were performed by 40,276 individuals (about 19.26% of active reviewers) and were linked to 32,869 manuscripts (35.55% of manuscripts considered).

Overall, reviewers decisions to sign reviews are distributed across journals and subject areas in a pattern similar to authors decisions to publish reviews. Signing rates were higher in public health, medicine, and related disciplines and less common in some branches of biology, as well as the physical sciences, earth sciences, engineering and math. Of articles with published peer reviews, 44% also had at least one signed review.

The clearest determinant of signed review was the reviewers recommendation (although reviewer recommendations are visible only to Academic Editors and the journal office). Signing rates were highest among reviewers recommending accept or revise decisions. For that reason, rates of signing on published articles (42.19%) are higher than for manuscripts generally.

At PLOS we have formal and informal conversations with our stakeholders and community members; we gather feedback in person and by email; we conduct surveys and structured interviews; we do meta-research; we observe.

Three years ago, PLOS chose an opt-in over an opt-out model of published review for a number of reasons. Our in-house editorial teams and editorial boards alike were committed to open peer review, and committed to ensuring it was only received positively by giving authors a real choice in the decision. More practically, PLOS serves more than 200 communities of research, each with their own needs and priorities. Open peer review is too important a development to force on those who have concerns and need to see a community embrace it first. Therefore, we chose to make openness easy. By leaving the door open, hopefully we can learn more from those who choose to walk through. And, its our hope that information like this can help inform the next round of discussions, innovations, and policies, leading ultimately to a more efficient, equitable, and trustworthy peer review system.

*Data from Web of Science, May 14, 2022

**All data current through May 9, 2022 unless otherwise noted. To be eligible for inclusion articles must have been submitted after May 22, 2019. Published Peer Review History is available at all PLOS journals, including launched in 2021, however only data from the seven long-established journals is included here.

Read more from the original source:
Published Peer Review History at PLOS: Observations from the past three years - The Official PLOS Blog - PLOS