Blog by Rob Bradley
As a Voice of Young Science (VoYS) member, I recently participated in an online Quality and Peer Review workshop, hosted by Sense about Science in partnership with IOP Publishing. At the
workshop, early career researchers (ECRs) discussed peer review and the quality of research with an expert panel and covered a wide range of topics, including:
- the fundamentals of peer review
- the role of peer review in maintaining research quality
- the systemic factors that affect research quality
- an insightful discussion about public engagement with peer review, and
- helpful advice for ECRs to get involved.
I was invited for the first time in my career to consider peer review as a research topic in its own right and hear directly from publishers, academic editors and researchers intimately involved in the peer review process.
As I reflect on this now, it seems bizarre that this should be the case. Peer review is supposedly foundational for modern science, providing confidence in published results to the research community and decision-makers. How can an ECR have undertaken 8+ years of academic training and never experienced formal training in such a fundamental topic? In this blog, I explore some of the challenges peer review that came up during the workshop and what we can do to improve research quality as a society.
What are the challenges facing peer review and research quality?
One of the most eye-opening parts of the workshop was the frank discussion of the strengths and weaknesses of peer review. I previously had no understanding of systemic issues impacting the quality of research, such as negative publication bias, where a journal favours a novel result over a negative one. It was also interesting to reflect on the different levels of transparency in the many types of the peer review.
For many of the ECRs and academics present, the source of the problems discussed, be it research that is not reproducible or poor-quality methods, could be traced back to a dilemma in research culture: the skewed incentives for academic success. In too many institutions, the number of papers produced by a researcher is the key metric by which their career progression is judged, otherwise known as 'publish or perish'. This leads to a situation where research quality and integrity can fall to the wayside because academics feel the pressure to publish frequently to maintain their funding or gain promotion. Other important contributions, such as promoting public engagement with research and performing peer review, are side-lined. Recognition for researchers who perform replication studies on the work of their peers is similarly limited. This is especially relevant alongside the issue of publication bias, leading to a skewing of the evidence-base.
Everyone has a role in addressing systemic issues that lead to poor-quality research
It is very easy to point to these issues and say that something needs to change. The benefits would be numerous. Addressing skewed incentives would allow researchers to focus on the quality of their research, meaning fewer resources wasted by institutions on sub-par research, a higher-quality evidence base available for decision-making in society and ultimately a healthier research culture. However, little has changed since the replication crisis was first declared over a decade ago. We need practical solutions to enact system-wide change. For example:
- Journals and funding bodies could promote quality research by requiring researchers to follow reporting guidelines, such as the EQUATOR Network's reporting guidelines for health research. Researchers can use these standards to ensure their reports include sufficient information to provide a complete account of their work in a transparent manner. If journals were to endorse similar guidelines for different fields, they would encourage greater consistency in reporting standards.
- Researchers could engage with initiatives such as Registered Reports, allowing for their methodologies to be reviewed prior to experimentation. If the protocol is accepted, then the results are published regardless of whether they are positive or negative, as long as the research follows the approved protocol. Registered Reports thus nip the problem of poorly planned protocols in the bud and reduce publication bias by moving the focus from positive results to methodology.
- Institutions could follow the lead of Utrecht University with its Recognition and Rewards programme. Utrecht announced in early 2021 that it was embracing Open Science as a core principle, meaning researchers will be evaluated based on their ability to work in a team, to promote transparency in science and to actively engage the public in their research.
Examples such as these demonstrate that pathways to a healthier research culture that emphasises quality and reliability do exist, and I hope to see the community embrace them.
ECRs should be equipped with the confidence and skills to engage with peer review
One, currently under-utilised but valuable, solution would be to include ECRs in the conversation about reliable research through training in peer review and quality. After my experience at the Quality and Peer Review workshop, it was clear to me that my training in this area was lacking. By encouraging ECRs to consider peer review as a core aspect of their contribution and skillset as a researcher, the attitude of the entire system could be altered. ECRs are responsible for the day-to-day execution of research proposals, but they often don't feel equipped to understand what goes into doing good-quality research or developing rigorous methods. If trained to consider the peer review step at the beginning of the research project, ECRs will feel more able to cast a critical eye over their own and their colleagues' protocols and ensure that quality, in particular robust methodology, is embedded from the start. Training would also inspire ECRs to feel more able to take part in peer review, where they can contribute a different and valuable viewpoint.
Current training programmes and workshops, such as those offered by IOP Publishing and Sense about Science, help ECRs understand the peer review system and the criteria for a high-quality review, providing an insight into a system that can otherwise seem intimidating or off-limits to ECRs. Sense about Science has also published the second edition of its free guide, 'Peer Review: the Nuts and Bolts', written by VoYS members for other ECRs. The guide explains the fundamentals of peer review and its role in society and was updated in 2021 to include recent changes in the world of peer review. I will take full advantage of the available peer review training, and I hope that peer review training receives more visibility and becomes a higher priority for the scientific community, considering its importance to trustworthy research practice.
As in many aspects of academia, one of the main obstacles facing ECRs is a lack of confidence. I believe training would be an important step in building confidence and would empower ECRs to engage in initiatives to revolutionise our current peer review system. It is up to us to shape the future of peer review, and I strongly encourage any ECR reading this to check out the resources highlighted in this article, sign up for the Voice of Young Science network to hear more about Quality and Peer Review opportunities, and take the first steps towards change.
About the Author
Rob Bradley is a Voice of Young Science member and a PhD student at Imperial College London, using mass spectrometry to investigate the oxidative stress protection systems employed by Pseudomonas aeruginosa. He is interested in research quality and the movement to update our current peer review system. If you are interested in joining the public discussion about research quality as an ECR and supporting your peers to do the same, then sign up for VoYS to get involved: Voice of Young Science - Sense about Science.
