In the early days of scientific societies (i.e. the 17th century), scientists would share their experimental results with each other at meetings, and receive feedback about their experiments in person. (The scientific journal wasn’t invented until later.) As the scientific community grew, it was impossible for everyone to be in the same room to hear about results, and so the amount of immediate feedback offered was limited to a few conferences or other gatherings. Recently, publishers, scientific societies and entrepreneurs have begun using the web to bring back the era of immediate feedback: so-called “post-publication peer review.”
One of the hallmarks of scholarly scientific publication is the review process. While it isn’t perfect, peer review is the process used by almost all scholarly publications to filter out bad-science, identify weak data analysis and make suggestions for better presentation of results. Traditional peer review was done before an article was published. In the age of print publication, this made the most sense because of the costs associated with printing.
Although most scientific journals are published online these days, peer review is still most often done prior to publication and the status of “peer reviewed” is held up to undergraduate students as an important hallmark of quality. Online publication makes post-publication peer review easier than ever – most often in addition to traditional pre-publication peer review.
Several new ventures are allowing scientists to comment on published papers, engage in discussions online and even publish more formal reviews. Informally, it started with blogs. Independent bloggers would read an article from the scientific literature and want to share their thoughts on it. As we have done on this blog, the blogger would cite the paper and discuss it’s results and any limitations. Perhaps the blogger might disagree with the original findings based on their own research. But the original author of the paper might never know that it was being discussed because there was no connection between the publishers version of the article and the blog post.
Next came user commenting. Taking a page from blogging platforms, publishers such as PLOS and BioMed Central experimented with allowing registered users to post comments on journal articles. This was only moderately successful. Some folks suggested that scientists were unwilling to directly criticize the work of others, although they might be willing to comment on a blog post discussing the article. While article comments kept the scientific dialog contained in one place, the conversations didn’t really take off, and some scientists I spoke with were skeptical about the idea of allowing anyone to comment on scientific articles. In recent years, third parties have started taking on the post-publication peer review mantle.
PubPeer allows users to make comments on almost any article that has a DOI, but the site is separate from publishers web pages. Scientists may be more willing to engage in frank criticism, but it is harder to connect the original paper to the comments. Importantly, PubPeer contacts authors when new comments are posted about their papers – in this way, PubPeer attempts to provide feedback directly to the authors that blogging doesn’t provide. PubPeer also attempts to preserve bling peer review by not publishing reviewer names.
PubMed Commons is the platform built into PubMed for using comments. Currently in a pilot phase, PubMed Commons invites authors of PubMed papers to join and comment on papers. Their names appear by their comments. By limiting who can comment on papers, PubMed Commons seems to want to reduce instances of non-experts commenting on papers. Articles related to politically controversial topics such as evolution, vaccines and climate change would theoretically be open to abuse if comments from everyone were allowed. Users will see these comments at the bottom of the PubMed entry for each article.
Open Review is the much hyped tool from the academic social network ResearchGate. Open Review encourages authors to publish a slightly more formal review than the simple commenting systems of PubPeer and PubMed Commons. Reviews are tied to your research gate account and to the entry for the article on ResearchGate.
As others have pointed out, the growing number of commenting systems produces a distributed system that can make it difficult for people to know where to go to get information about an article. And I suspect that these systems will only multiple in the near future. These new tools are providing a way for scientists to make their private conversations about recent research public, and allowing them to expand the group of people they can chat with. Although the goal is to provide feedback and commentary, the limits of space (how many people can fit in the room, who has the office next door) no longer apply.
New scientists have grown up commenting on their friends pictures, their silly comments on Facebook and their favorite YouTube videos. Will this practice carry over into their scientific publishing? Will publishers eventually link back to these commenting systems? Will database producers (e.g. Scopus, Web of Science) incorporate user comments into their systems? It will be interesting to find the answers to these questions and see how these systems impact scholarly scientific communication.
Leave a Reply