PostPub: Help Scientists Reproduce

Scientific understanding progresses through incremental steps and experimental results have to be reproduced after initial publication, particularly in the variable world of the life sciences.  Often we feel that post-publication process currently lacks coherence, resulting in unnecessary duplication of work and frustration for experimentalists. Here is why and how we can help.

The first step in the discovery process is to plan and perform a set of experiments that support a hypothesis. Based on those experiments, a paper is submitted to a journal for review.  At the journal, the process of peer-review is the gold standard to assess a paper's scientific merit. If the paper is published, the discovery is read and assimilated.

Here’s the catch. In the competitive world of ‘get your paper out ASAP’, many scientists do not have the luxury to confirm a discovery from every angle.  It is possible that a published results may not hold true for certain circumstances. Hence the need for other labs to reproduce a published finding to determine if the result is a general or specific truth.  However, independently repeated experiments confirming, coloring, or contradicting published work are often not published. This is where we would like to help.

Several years ago, a group of scholars sitting at brunch at the Snug in Cambridge, England discussed the issue.  A lot of biological findings were published everyday, and not all of it could be taken at face value because biology was variable and there was no way of knowing if experiments were reproduced elsewhere.  This information is crucial for young scientists.

We decided that it would be good to build a repository for data of published experiments.  In an effort spanning Cambridge, Yale, UCL, and Columbia, we designed PostPub, a platform that provides identity-verified, results-driven, and experimentalist-centric platform focusing on reproducibility. 

We hope this venture will involve experimentalists in the post-publication (PostPub) validation phase. Currently, comments and questions on a publication are directed towards the senior 'corresponding author' or Principal Investigator in a lab, who often does not have the time, patience or indeed direct involvement to answer technical details on methodology of experiments.

Our solution works as described in the scheme above. You repeat a published experiment and obtain different results; this could mean either that you, most likely, have technical issue or the original publication is not generalizable.  PostPub provides a platform where you can present your data and methodology, which is 'unpublishable' since it is a repeat, and a link to the original publication.  Others interested in the same topic, perhaps even the experimentalist in the primary publication, notified through PostPub, can reply to your version, clarifying the difference.  Other members of PostPub can comment, if they have data that are relevant. A discussion thread is built that could be valuable to anyone in the field as much as the original publication. By amalgamating discussion threads, we generate a Reproducibility Index (RI) which measures a paper's reproducibility on a scale of yay (green), maybe (yellow), and nay (red). The verdict or Cumulative Reproducibility Index judges a paper's generalizability with a verdict of yay, maybe, and nay alongside the number of discussion threads.

Read about PostPub's Team, Frequently Asked Questions, and Press.

We hope to improve PostPub with your suggestions.

We can be reached at team@postpub.org