The recording for the Open Research Forum is now available to watch via Stream (University of Reading members only):
On 18th May, we held our most recent Open Research Forum on open peer review. We were delighted to be joined by a range of speakers who contributed fascinating and thought-provoking perspectives and insights into the world of open peer review.
We kicked off our session with a quick poll to capture the initial thoughts of our attendees. We asked whether they were in favour of open peer review. The results of this are shown below:
We were interested to see whether people’s attitudes would change during the session.
The Future of Peer Review: Four Schools of Thought
Presented by Stephen Pinfield, Professor of Information Services Management/Associate Director of the Research on Research Institute, University of Sheffield
The Research on Research Institute (RoRI) is an international consortium of researchers, research funders, and other key stakeholders. It aims to co-design and co-produce research projects together with stakeholders in the research system, providing evidence-informed insights that can be translated into practical solutions.
A recent RoRI report on the future of peer review identified open and transparent peer review as an area of significant innovation. It identified four schools of thought regarding peer review which offer different perspectives and highlight aspects of peer review which are problematic.
The Four Schools are:
- Quality and Reproducibility
- Democracy and Transparency
- Equity and Inclusion
- Efficiency and Incentives
Whilst the Schools complement each other in some areas, there are significant tensions between them. Conversations between schools are suggested to provide creative ways to resolve these. There needs to be more heterogeneity in the peer review system, allowing for co-existence of different forms of review. The publishing peer review system needs to be aligned with broader developments in the research system. Finally, a requirement was identified for the research on research community to develop a rigorous evidence-informed understanding of the peer review system.
Open Peer Review at F1000Research
Presented by Eleanor-Rose Papas, Editorial Operations Manager and Peer Review Manager, F1000Research
F1000Research and associated publishing platforms, including Wellcome Open Research, Gates Open Research, and Open Research Europe are platforms that aim to change the way science is communicated through the open and transparent publication of articles, associated data and peer review reports.
Since 2012, F1000Research have offered an open and transparent review model. The basis of this is that articles are published first before being subjected to open peer review. This has the benefit of increasing dissemination, with authors able to revise articles according to the feedback they receive. All peer review reports are available to read, with reviewers acknowledged on the report. This means that the reviews can become part of academic literature, so are assigned DOIs and are data minable.
The review process is formal and invited, with reviewers provided with a code of conduct that they must adhere to. Therefore, F1000Research can preserve integrity, ensuring that readers can trust the peer review process.
eLife’s mission and the PRC model
Presented by Ailís O’Carroll, eLife Community Manager
eLife is a not for profit peer-reviewed open access scientific journal for the biomedical and life sciences. Its mission is to accelerate discovery by operating a platform for research communication that encourages and recognises the most responsible behaviours.
eLife’s ‘publish, review, curate’ model involves researchers uploading a preprint to an appropriate server. The preprint will be subjected to consultative peer review with an evaluation summary provided. Public Reviews are posted alongside the preprint, with a response from the authors if they choose to produce one. Authors then have the option to transfer to another journal or to revise and publish with eLife. Public Reviews are also offered with a wider public audience in mind.
For this new system to work, eLife emphasizes the need for the research community to become involved. They work alongside PREReview, Sciety and eLIFE Community Ambassadors to improve the peer review process.
Open Research Forum on open peer-review- A personal account
Presented by Rémi Tailleux, Associate Professor of Physical Oceanography, Department of Meteorology, Open Research Champion for MPCS
Remi discussed his experiences of open peer review as both an author and a reviewer for Copernicus journals and Nature Communications.
Remi highlighted the positives of open peer review. The paper is immediately available and has a DOI, meaning that it can be cited even if it has not yet been accepted for publication. The open discussion tends to discourage unprofessional reviews. Authors are also able to receive a wider range of comments, as anybody can comment on papers. This can help authors to receive greater feedback, resulting in a better paper.
Despite this, Remi discussed how in his experience, open peer review can lead to superficial responses. Reviewers and authors must take care in monitoring the responses they provide, as they will remain public forever. Traditional models of publishing enable editors to moderate exchanges between authors and reviewers. Editors in some models only intervene at the end of the open discussion, playing less of a filtering role than in the standard model.
Open Peer Review: A Journal Editor’s Perspective
Presented by Angelique Chettiparamb, Professor of Urban Planning and Governance, Henley Business School (Real Estate and Planning), Managing Editor: Planning Theory
Angelique Chettiparamb- Slides
Planning Theory is a high-ranking journal in spatial planning. The editorial board decided to join the Web of Science Transparent Peer Review (TPR) Program through their publishers-Sage Publishing-on a pilot basis. The board decided to review the initiative after one year of implementation and then evaluate whether the journal should continue with the program.
Authors could choose to opt in for TPR at submission. Reviewers had to opt in for the process, as opting out would have meant that the entire content for the peer review process of the article would not be published. However, reviewers could preserve their anonymity, as they had a choice to do so.
Since TPR came into operation on 1st April, statistics from Sage Track show:
- 6 new submissions- all authors have opted in
- 2 first revision submission- authors have opted in
- 1 second revision submission- author has opted out of TPR
- All reviewers have opted out of sharing their identity.
There was enthusiastic support from some participants, as TPR aligns with principles of openness and transparency. However, there were questions raised of whether TPR improved reviews in terms of providing constructive and substantial feedback. Some members of the editorial board were unsure what TPR was aiming to solve. There were concerns surrounding post-publication exercise of power, authors possibly refusing to review, and reviewers writing for an audience rather than producing a substantive review. However, open peer review acknowledges the contribution that reviewers make to a manuscript. This was suggested as challenging authorial ‘glory’ since publishing the review process shows how reviewers’ inputs can shape the final published output.
Panel Discussion/Q&A
We invited our speakers to contribute towards a panel discussion on open peer review. The discussion first focused on the unintended consequences of the process such as exercises of power and the quality of reviews.
The panel also reflected on changes or interventions within the wider ecosystem that would encourage a greater adoption of open peer review models. This included a spectrum of balancing concerns and incentives, such as enabling authors to preserve their anonymity and promoting the contribution reviewers can make to the publishing process. Thus, incentive systems need to be aligned to facilitate the open peer review process. Currently, review is a voluntary contribution from researchers as it is the expected activity of the academic community.
To inspire a cultural shift, aspects such as paying reviewers were touched on. However, this does not resolve wider issues, as there is a need for institutions to recognise peer review more as a scholarly output. Training researchers how to write good peer reviews could contribute towards this. Training sessions and structured mentoring programmes could help, as well as early career researchers receiving guidance from their supervisors.
After hearing from our speakers, this was the end poll:
It was fascinating to see how the talks influenced our audience’s attitudes towards open peer review.
RSS - Posts