It is how we know scientific findings are valid: if an experiment produces a result once, then it will likely produce it a second time, and a third, and so on – each time the experiment is repeated. However, some research studies produce results that cannot be reproduced.
Making research procedures more robust will underpin public trust in biomedical science, as well as leading to improved products, services – and health – for everyone. This is why, today, 46 IAP for Health member academies commit to taking action to address issues of reproducibility in a Statement: ‘A Call to Action to Improve the Reproducibility of Biomedical Research’. The Statement outlines the causes of irreproducible results and recognises the leading role the academies can play in addressing this challenge in their own countries and worldwide.
Results may be irreproducible simply because of natural variability in the biological systems being studied. Alternatively, they might be caused when scientists do not accurately report the details of how their experiment was carried out for others to follow; because the experimental designs were flawed; or even because the data produced were not analysed robustly enough.
Whatever the reasons, this lack of reproducibility in research is creating growing unease among scientists and the public, who are calling for more to be done to ensure that the quality of evidence is as high as it should be – including evidence from biomedical research.
Evidence of the scale of the reproducibility challenge is compelling. The ‘Reproducibility project: psychology’, for example, which published its results last year, set out to reproduce 100 psychological studies and found that just 36% of its replication attempts produced meaningful results, while 97% of the original studies claimed to have done so. Other similar findings have emerged from fields such as cancer biology and drug development.
The academies highlight that research that cannot be reproduced wastes resources and delays the translation of results into clinical applications. It also has the potential to undermine the public’s trust in research findings, which threatens the reputation of biomedical science.
Lai Meng Looi, of the Academy of Sciences Malaysia and co-chair of IAP for Health, says: “The issue of irreproducible results affects researchers in all countries – high-income countries and low- and middle-income countries. By releasing this Statement, we hope that IAP for Health and its member academies, representing the scientific community, will be able to raise awareness of the issue. We expect that our member academies will now present it directly to their national governments and disseminate it through their national scientific communities so that our recommendations can begin to be implemented.”
IAP for Health’s other co-chair, Detlev Ganten of the German National Academy of Sciences, Leopoldina, added: “Like all IAP Statements, this Statement on reproducibility has been thoroughly reviewed. It presents – based on the latest evidence from our member academies – the best impartial advice to policy-makers, whether at national or international level, as well as to healthcare and biomedical researchers, scientific publishers and funding agencies”
Among its recommendations, the Statement urges a shift in the culture of science to better reward robust scientific methods and, when it comes to career advancement, to value the development of good research practice as much as new findings.
The Statement recommends that funders set the tone for reproducible research by, for example, rigorously assessing proposed experimental designs; and urges publishers and journal editors to enable greater openness and transparency by publishing replications of previous research or negative results. The Statement also encourages academies and other members of the scientific community to take steps to ensure that peer-review practices focus on the quality of the science more than on any potential excitement generated by the results.
Finally, the IAP for Health Statement encourages scientists to participate openly in the global dialogue around how to improve the reproducibility of results throughout the international research environment.
"Irreproducible results are often preventable and avoidable," adds Dorothy Bishop, Professor of Developmental Neuropsychology at Oxford University, UK, and chair of the IAP for Health working group that prepared the Statement. "Now that these problems are widely recognised, we have an opportunity to unite and address the key areas highlighted in the IAP Statement. The motivation is there for a concerted effort from academies, the scientific community, funders, institutions and publishers across the world to strengthen the robustness of peer-reviewed medical research and maintain confidence in its findings."
The Statement, “A Call to Action to Improve the Reproducibility of Biomedical Research” was released during the IAP for Health Conference on ‘Promoting Health’, in Beijing, China (27-28 September 2016).
The Statement will be available at: tinyurl.com/IAP-Reproducibility-Statement
About The InterAcademy Partnership for Health
IAP for Health is a component network of the InterAcademy Partnership. Its membership comprises 78 academies of medicine or academies of science and engineering with medical sections. It is committed to improving health worldwide, including through the release of consensus statements on matters of importance to global health. IAP for Health Statements such as this one are prepared by a working group comprising experts nominated by member academies, and are released once they have been endorsed by more than half the member academies of the network.
www.iamp-online.org
For additional information, contact:
Peter McGrath
IAP Coordinator
Trieste, Italy
[email protected]
Sean Treacy
TWAS Public Information Office
Trieste, Italy
[email protected]
For press enquiries in the UK:
Giorgio De Faveri, PhD MCIPR
Senior Press Officer
The Academy of Medical Sciences
41 Portland Place
London W1B 1QH
[email protected]
Direct dial: (+44) (0)20 3141 3206
Mobile: (+44) (0)78 8590 3528
