Is the peer-review system broken? A look at the PLoS ONE paper on a hand designed by “the Creator”

abstract from PLoS ONE paper god creator hand design

This is the abstract from the paper with the relevant phrase highlighted.

The internet has recently gone nuts over a scientific paper published in PLoS ONE (a generally respectable journal) which contained several lines suggesting that the human hand was designed by “the Creator.” The paper was quickly retracted, but the brouhaha continues, so I want to briefly talk about the controversy and what it can teach us about the scientific process and the peer-review system.

What was the paper?
The paper in question is “Biomechanical characteristics of hand coordination in grasping activities of daily living” by Lui, Xiong, and Huang. The paper talks about how the hand’s structure and biomechanics allow it to function in a highly versatile way, and the science seems okay, except that several places make really bizarre jumps to divine conclusions. For example, “Hand coordination should indicate the mystery of the Creator’s invention.” When I read lines like that, my instant thought was, “this has to be a translation error.” The paper was clearly not written by native English speakers, and the references to the “Creator” were so jarring and out of place that it seemed like surely it was a mistake. Indeed, the authors have now stated that it was.

We are sorry for drawing the debates about creationism. Our study has no relationship with creationism. English is not our native language. Our understanding of the word Creator was not actually as a native English speaker expected. Now we realized that we had misunderstood the word Creator. What we would like to express is that the biomechanical characteristic of tendious connective architecture between muscles and articulations is a proper design by the NATURE (result of evolution) to perform a multitude of daily grasping tasks. We will change the Creator to nature in the revised manuscript. We apologize for any troubles may have caused by this misunderstanding. We have spent seven months doing the experiments, analysis, and write up. I hope this paper will not be discriminated only because of this misunderstanding of the word. Please could you read the paper before making a decision.

Nevertheless, because of the uproar of the scientific community, PLoS ONE has decided to retract the paper. Personally, I think that full retraction is unnecessary. Anyone who reads this blog knows that I am not in anyway a creationist, and I expect science to be held to a high standard. Further, I do think that this error represents a serious mistake on the part of the editors and reviewers. Nevertheless, this does appear to be an honest mistake by the authors, and the science of the paper seems sound. So my personal opinion is that the paper should be revised to correct the translation issues, sent back out to new reviewers who will double check the science, then a decision should be made about rejecting or accepting it based on those reviews. I know how much work goes into doing research and writing a paper, so it seems like a shame to have that go to waste just because of a translation issue. Further, given that the authors are not native speakers, I think that most of the blame for this lies with the reviewers and editors, not the authors.  Nevertheless, let’s continue to examine the issue further.

What’s the big deal?
Some people may be wondering why this is even important. So what if a paper mentioned “the Creator”? Further, I can easily see many Christians responding with outrage and insisting that this is evidence that scientists are all closed minded atheists who are setting out to prove that God doesn’t exist (this is a common an irrelevant attack that they use to dismiss science).

The reality is that this actually has nothing to do with scientists being atheists (many aren’t); rather, it has to do with the nature of science itself. One of the fundamental guiding principles of science is that we live in a natural world that is governed by natural laws. Let me clear, this is not an inherently atheistic viewpoint. Rather, it is a necessary starting point for scientific inquiry. You see, if we don’t view the world as a natural system that is governed by natural laws, then studying it becomes futile because we can invoke God anytime that we want for anything that we want. Anytime that we have an unexplained phenomena, we can just plug God in as the answer and move on. Indeed, if nature is being directly governed by God rather than the laws of physics, then there is really no point in even studying it, because we will invariably get to a point where there is no scientific answer.

This leads to the next major problem. Namely, science is inherently incapable of answering questions about the supernatural. The type of conclusion that the authors’ mistake suggested is a type of conclusion that science simply can’t arrive at. Science is, by definition, the study of the physical universe, whereas God (if he exists) would inherently by supernatural, and science can’t address questions about the supernatural. Thus, the conclusion, “God did it” is a conclusion that science isn’t capable of reaching.

Therefore, the strong reaction of scientists isn’t evidence that all scientists are atheists who are angry at the concept of God. Rather, scientists reacted strongly because this type of reasoning is a fundamental assault on the very nature of science.

Is peer-review fundamentally flawed?
The peer-review system is not perfect. Let me say that again, the peer-review process is an imperfect system and bad papers do sometimes get through. Ultimately, the system relies on humans, and humans are flawed. Thus, flawed papers sometimes get published, but that doesn’t mean that the system is worthless.

In this particular case, I’m truly dumbfounded about how this paper got through, because one of the references to the “Creator” is in the abstract. Here’s the important catch though: it’s easy to point out examples where the system clearly failed, but you also have to consider how many times the system worked. In other words, we see the glaring examples of flawed papers that made it through the review process, but what we don’t see are the countless thousands of papers that didn’t make it through.

To put this another way, given the millions of scientific papers that get submitted to journals, it is inevitable that a few bad ones will get through; however, it would be a huge mistake to tout those examples as evidence that the system doesn’t work, because that would ignore all of the times that the system did work. Editors and reviewers get overworked, and sometimes they get lazy,  but that’s not the norm. Most editors and reviewers do a very thorough job of reviewing papers. I have personally sent multiple papers through the review system, and I usually get back an extremely lengthy list of comments that critiqued every aspect of my paper, and a mistake like this would have been caught before the editor even sent the paper out for reviews. So you should not view this type of mistake as being normal.

The system worked!
If you try to use this paper as evidence that the peer-review system doesn’t work, then you are missing the fact that the system actually did work! As I’ve often argued, peer-review doesn’t end when a paper is published. Rather, the paper will be critiqued by other researchers who read it, and if serious mistakes are found, they will contact the editors and the paper will be retracted. That is exactly what happened here. Other scientists spotted the mistakes that the editors/reviewers missed, they brought attention to the issue, and the paper was retracted. So, rather than providing evidence that the peer-review system is fundamentally broken, this is a great example of the system working. Yes, there was a very serious mistake at one stage the process, but the other stages corrected that mistake. That is one of the best things about science: it is self correcting.

In short, yes, this was a serious mistake. It is a huge indictment on PLoS One, and they should take a very long hard look at their editorial staff and policies. However, this example and others like it do not prove that they system as a whole is broken beyond repair. Yes, the system is imperfect, and yes, mistakes do happen, but overall, the system works and it both prevents many bad papers from being published and often removes the bad papers that slipped through the formal review process. The great thing about science is that if you make a mistake, there are thousands of other researchers you are ready and willing to tear your work to shreds.



This entry was posted in Nature of Science and tagged . Bookmark the permalink.

2 Responses to Is the peer-review system broken? A look at the PLoS ONE paper on a hand designed by “the Creator”

  1. Max says:

    English is the third language I have learned so far and when I first heard words and phrases such as “intelligent design”, and “creationism”, I didn’t even in my dreams imagine it had anything to do with god. Especially when I heard the phrase “intelligent design”, I thought it had to do with design and engineering. We have smartphones. I thought maybe we can use “intelligent design” to make intelligent phones which are more advanced that smartphones. Only later when I learned the meaning did I realize religious people were trying to sound more credible by using such terms. Looks like I wasn’t the only one who was fooled by such terminology.


Comments are closed.