Unfortunately, bad papers sometimes get published, and those faulty results often get hailed by members of the anti-science community as evidence for their positions. As a result, it is extremely important to both look at the entire body of literature on a topic and critically examine the papers themselves. More often than not, when you look at the literature for a scientific topic, you will find that most studies have converged on a consistent conclusion, while a few outliers have reached the opposite conclusion, but those outliers are usually riddled with problems and were published in minor journals. Thus, it is foolhardy to latch onto the handful of papers that agree with you, while disregarding the vast majority of papers that disagree with you.

This is a prevalent problem and one that I write about frequently (for example, I have previously written about how to evaluate scientific papers, Tenpenny’s cherry-picked “vaccine library,” the supposed lists of papers showing that vaccines cause autism, etc.). In this post, however, I am going to focus on GMOs. I’ve talked about this before, and explained that the anti-GMO position is, in fact, a form of science denial that is based on ideology, not evidence. As with topics like vaccines and climate change, the evidence that GMOs are safe for both humans and the environment is overwhelming, yet activists rally around the tiny subset of papers that agree with them. When you critically examine those papers, however, it quickly becomes clear that they are junk science. Indeed, that was the conclusion of an intriguing new review that was published in the journal Plant Biotechnology Journal, and I want to spend a few minutes talking about it.

Note: This paper was specifically on human health implications, not environmental implications, but the same story holds true when you look at the environmental papers.

The paper in question is titled, “Characterization of scientific studies usually cited as evidence of adverse effects of GM food/feed,” and I encourage you to read the whole thing. It is very accessible and easy to follow. Nevertheless, I will talk about a few highlights. In a nutshell, this study reviewed the literature, identified 35 studies that reported negative health effects associated with GMOs, then it evaluated the quality of those studies and placed them in the context of the wider literature. Unsurprisingly, it found that the quality of most of those 35 studies was quite low, and they often contained blatant flaws.

Small proportion of studies

First, it is important to note that these 35 studies represent a tiny fraction of the literature (only around 5% of the GMO papers the authors were able to identify). Right of the bat, that is a huge red flag. If the results of those studies were correct, then they should be what the majority of studies are finding, not what a tiny minority are finding. Indeed, because of the way that statistics work, we expect about 5% of experiments to produce false positives just by chance (details here). So even if these studies were flawless, they would still be indistinguishable from statistical noise when you look at the entire body of literature.

It is also worth mentioning that an extensive review of the literature, that looked at both human health and the environment, examined 1,783 studies and concluded that GMOs are no worse than conventional crops for humans or the environment and in some cases they are better (Nicolia et al. 2013).

Few labs and authors

The next thing to note is that these 35 papers were produced by a handful of researchers. Indeed, one researcher was an author on 11 of those papers. So what you have is a few labs that are repeatedly publishing papers that support the previous findings of those same labs. This is another problem. If their results were real, then other independent scientists from around the world should have found corroborating results, but they haven’t. That strongly suggests that something wrong is happening in these few labs, and, indeed, in some cases, there is clear evidence of fraud (more on that in a minute).

Low ranking journals

When evaluating a paper’s claims, it is always a good idea to consider the quality and reach of the journal that published it (this is often measured by an “impact factor” which is based on how widely cited a journal’s publications are). Whenever you have a really important, novel result, you try to publish it in a high impact journal. In contrast, if you have a fairly uninteresting result that everyone already expects or a result that is very specific to a narrow field, you generally publish it in a low-ranking journal. Thus, if the science is solid for claims like, “GMOs cause cancer” then you expect those papers to appear in very high impact journals, and you should be very suspicious when they show up in tiny journals that no one has ever heard of. Ask yourself, “why wasn’t a result that is this important and interesting published in a high ranking journal?” The answer is usually that it couldn’t pass their standards.

So, getting back to this list of 35 papers, what did the authors find? Perhaps unsurprisingly, nearly all of those papers were in minor journals. Indeed, eight of those papers were published in journals that are so minor they don’t even have an impact factor (that is another huge red flag), and an additional six had an impact factor less than one (which is a really low impact factor). In fact, only one of those 35 papers (Ewen and  Pusztai 1999) was published in a high ranking journal. This paper was, however, the source of great controversy. One of its reviewers found that it was flawed and should not be published, and another expressed serious doubts over the paper, but thought that, for the sake of openness, it would be best to publish the paper and let the general scientific community evaluate it, rather than risking the appearance of a conspiracy or cover-up (see the article for more details). I personally disagree with that decision, but it is, nevertheless, evidence that the crazy conspiracy theories about scientists suppressing evidence are just as insane as they sound. Additionally, The Lancet (the journal that published it) also published an editorial stating that some of the reviewers took issue with the paper.

Note: It’s worth mentioning that the review I am talking about was published in a well-respected journal with an impact factor of 7.443.

Conflicts of interest

I’m not personally super concerned over conflicts of interest (e.g., funding sources and employment by companies or activist groups), but it is, nevertheless, worth mentioning them. They found that 21 of the papers (60%) had conflicts of interest, which is higher than than the rate of conflicts of interest in the general body of GMO literature (Sanchez 2015 found that 58.3% of GMO studies had no conflicts of interest, 25.8% had clear conflicts of interest, and the remaining 15.9% could not be assessed [i.e., the authors were not linked to companies, but did not declare their funding sources]). The only point that I really want to make here is that this isn’t a situation where all 35 papers are free from conflicts of interest and all the papers saying that GMOs are safe are loaded with conflicts of interest. Rather, you have some of both in each group, which leaves you with 14 anti-GMO papers that have no conflicts of interest and at least 406 pro-GMO papers that have no conflicts of interest (see original paper for details).

Note: The authors of the review paper did acknowledge that they themselves have conflicts of interest, but that does not invalidate their results, and it does not give you carte blanche to ignore their findings. As always, when a conflict of interest is present, you should apply greater scrutiny, but you should not blindly disregard the study.

 Problems with the papers themselves

Finally, and most importantly, the authors found that problems abounded with the studies themselves. They summarized this nicely in table 1 (as well as providing more details in the text), but the problems included things like, “Flawed statistics (fishing for significance)” (de Vendomois et al. 2009), “No use of non-GM soybean as control” (El-Kholy et al. 2014), “No information on crop source; inadequate sample size” (Yum et al. 2005), “No biological relevance” (Tudisco et al. 2007), etc.

Further, the review talks about some of the more well-known examples of flawed GMO research. For example, there is Seralini’s infamous rat study which was so flawed that it was retracted (Seralini then submitted it to a predatory journal where it is currently published). Similarly, there are multiple papers by Federico Infascelli. If you pay attention to news in science at all, then his name might sound familiar, because last year it was discovered that he had manipulated the data on at least two of his papers, resulting in both of them being retracted. His entire body of work is now under close scrutiny and his reputation has been forever tarnished (more details at Retraction Watch and Science-Based Medicine).

Finally, it is worth mentioning that this is not the first paper to address this issue. A previous study (Panchin and Tuzhikov 2016) also found that anti-GMO papers were full of problems (namely, statistical problems), and that when you used the correct statistical tests, the reported negative effects of GMOs vanished.

Conclusion

So where does this leave us? The answer seems pretty clear: anti-GMO studies represent a tiny portion of the literature, they are usually published in low-quality journals, they are riddled with statistical and methodological problems, several of them have been retracted (sometimes because of scientific fraud), and they are refuted by a vast body of literature. Further, before you baselessly suggest that the pro-GMO papers were all bought off by big companies, please note that less than half of the general body of GMO literature contains conflicts of interest, whereas 60% of the anti-GMO papers contain conflicts of interest. In short, the anti-GMO papers are, at best, statistical noise, and they do not, in any way shape or form represent compelling evidence that GMOs are dangerous. Most of them are junk science and should be rejected as such.

Note: A similar paper on climate change papers reached the same conclusion. Namely, the handful of papers arguing against anthropogenic climate change are filled with problems. (Benestad et al. 2016; the supplemental information is particularly useful)

Literature cited

• Benestad et al. 2016. Learning from mistakes in climate research. Theoretical and Applied Climatology 126:699–703.
• de Vendomois et al. 2009. A comparison of the effects of three GM corn varieties on mammalian health. International Journal of Biological Sciences 5:706–726.
• El-Kholy, et al. 2014. The effect of extra virgin olive oil and soybean on DNA, cytogenicity and some antioxidant enzymes in rats. Nutrients 6:2376–2386.
• Editors of the Lancet. 1999. Health risks of genetically modified foods. The Lancet 353:1811.
• Ewen and Pusztai. 1999. Effects of diets containing genetically modified potatoes expressing Galanthus nivalis lectin on rat small intestine. Lancet. 354:1353–1354.
• Nicolia et al. 2013. An overview of the last 10 years of genetically engineering research. Critical Reviews in Biotechnology 34:77–88.
• Panchin and Tuzhikov 2017. Published GMO studies find no evidence of harm when corrected for multiple comparisons. Critical Reviews in Biotechnology 37:213–217.
• Sanchez 2015. Conflict of interests and evidence base for GM crops food/feed safety research. Nature Biotechnology 33:135-137.
• Sanchez and Parrott 2017. Characterization of scientific studies usually cited as evidence of adverse effects of GM food/feed. Plant Biotechnology Journal.
• Tudisco et al. 2007. Investigation on genetically modified soybean (Roundup Ready) in goat nutrition: DNA detection in suckling kids. Italian Journal of Animal Science 6:380–382.
• Yum et al. 2005.  Genetically modified and wild soybeans: and immunological comparison. Allergy and Asthma Proceedings 26:210–216.