NIH Launches $174M Replicability, Reproducibility Initiatives

The National Institutes of Health is planning a five-year, roughly $174 million effort to encourage more scientists to test other researchers’ published findings.

The agency is a massive funder of university biomedical research. In an editorial Thursday in Science, NIH Director Jay Bhattacharya discussed the need for the Rigor, Replicability, and Reproducibility to Promote Excellence, Accuracy, and Translation in Science program—or R3PEATS.

“New discoveries naturally attract the most attention within and outside the scientific community, whereas studies seeking to confirm already published work rarely do,” Bhattacharya wrote. “The latter includes research that tests scientific questions in different ways to determine whether findings are robust and broadly applicable (reproducibility) and research that repeats studies using the same methods and conditions to determine whether the same results can be achieved (replicability). As a result of this attention imbalance, substantial time, talent, and resources can be consumed chasing findings that may not hold up under further evaluation.”

Bhattacharya has repeatedly said he wants NIH to do more to promote replication studies. He’s floated starting an in-house journal to publish them, though it hasn’t yet appeared. But the new initiatives could lead to an increase in the number of publications that confirm or debunk previous findings.

R3PEATS will comprise four initiatives to study “practical ways to incentivize and support rigorous and replicable research,” according to a document NIH provided Inside Higher Ed on Friday.

The program plans to fund, among other things, projects in which two to five teams each “will jointly address the same scientific question(s) using complementary methods and outcome measures to generate a fuller biological understanding than a single study usually provides.”

NIH also plans to offer paid fellowships for “rigor ambassadors” and to provide funding to “establish consortia of two to five institutions each” that will build programs providing “training, infrastructure, mentorship, and resources for at least four appointed scholars per award.” The agency also intends to establish an eventually self-sufficient U.S. Reproducibility Network, modeled after one in the United Kingdom.

NIH also publicly launched Thursday a Linked Discoveries tool in its National Library of Medicine’s PubMed research paper database. In a news release, the agency said the tool uses an “AI-informed approach” to identify publications related to an article a user is interested in, providing graph and timeline views to explore connections.

“Researchers can narrow a neighborhood [of papers] by conditions, genes and chemicals and see information such as citation connections, reviews, retractions and NIH-funded publications,” the release said. That neighborhood might show whether any replication studies have been done on particular publications.

In his editorial, Bhattacharya called on the academic community to help in this effort to ensure that scientific publications are reliable.

“Research institutions should extend concerns to examine whether hiring, promotion, and recognition reward rigor as well as novelty,” he wrote. “Publishers should create venues for replication and reproducibility studies. Funders should recognize that independent verification is not secondary to discovery; it is part of what makes discovery useful. Scientific leaders should make clear that correcting or refining the scientific record is an achievement.”

Heather Pierce, senior director for science policy at the Association of American Medical Colleges, said study reproducibility has been discussed for a long time. She said all these efforts are trying to address a bigger question: “Do research results translate into clinical practice or improvements in health in ways that matter?”

The important thing, she added, is “less about discovering which studies are wrong and more about just continually refining our knowledge through the scientific process.”