Naturopaths use deceptive tactics to support pseudoscience

Earlier this year, a new review paper was published claiming to show evidence that naturopathy was effective (Myers et al. 2019). I’m a bit late to the game on this one, but I still want to briefly talk about this review, because it is a good illustration of the deceptive tactics people often use to claim that quack treatments are effective.

The paper in question is, “The state of the evidence for whole-system, multi-modality naturopathic medicine: a systematic scoping review.” The first thing to note is that it was published in The Journal of Alternative and Complementary Medicine. It is hardly surprising that a journal dedicated to alternative medicine is going to publish a paper claiming that alternative medicine works. In other words, this review was not published in a respected medical journal. Rather, it was published in a biased journal with a reputation for research that is questionable at best. This is a big red flag.

Nevertheless, let’s look at how they set up the review, but before getting to the review itself, I want to use an analogy that will help to illustrate the issues with it and with naturopathy more generally. I’m a big fan of analogies because they help to reveal underlying problems in reasoning.

So, for this analogy, let’s imagine that I have started a new type of medicine called electroism, in which I argue that electrical imbalances in your body are the cause for most diseases, and to cure these issues, you need to eat healthy, exercise, and periodically electrocute yourself with a 12-volt battery. Now, imagine that multiple studies look at the effects of electrocuting yourself with a car battery and conclude that it is not a good idea. I then respond by saying that those studies are invalid because you can’t look at just one piece of electroism in isolation. You have to look at the whole system together, and when you do that, electroism is beneficial.

Do you see the problem? Do you see how I have deceptively manipulated things to make it look like electroism works? My ridiculous electroism method includes both utter nonsense (electrocuting yourself) and things that are known to cause improvements and are fully embraced by the medical community (i.e., diet and exercise). Thus, by insisting that electroism can only be studied as a whole, I can make it look like electroism is beneficial, because I’ve now included things that do work alongside my nonsense. In other words, if you make a Venn diagram of actual medicine and electroism, there will be a zone of overlap containing diet and exercise, and by insisting that electroism has to be taken as a whole, I can deceptively use the benefits from that zone of overlap as evidence that electroism as a whole works.

My example may seem silly, but that is precisely what naturopathy does and what this study did. You see, naturopathy is very broad and involves a few good things and a whole lot of nonsense, but naturopaths (and the study authors) insist that it has to be looked at as a whole. As a result, the benefits of the good parts (which are also in actual medicine) make the nonsense parts look good.

Let me explain that in more detail. As the study states, “The [World Naturopathic Federation] defines the naturopathic profession based on two fundamental philosophies of medicine (vitalism and holism).” Holism is a semi-scientific concept (depending on how it is applied). It is the notion that we should treat the whole person and consider factors such as mental health and socioeconomics when treating someone. In some cases, this is fine, and is often a part of mainstream medicine. Stress and other lifestyle factors can lead to medical issues and I have no inherent problem with taking those into account during medical diagnoses. This is admitadly an anecdote, but I once went to a doctor (a real doctor) with digestive issues, and after running some tests and talking to me, she determined that I was probably just stressed to the point of making myself physically sick (she was right). This does not, however, mean that all medical issues are being influenced by multiple factors. Sometimes an infection is just an infection, which is where holism starts to get into trouble.

Vitalism is far more problematic. This is the notion that living things are somehow fundamentally different from non-living things and living things have a vital life force which can affect health. This is where all manner of “energy” based treatments start to come into play, and the whole thing is pre-scientific hogwash. There is no evidence or logical reason to believe that there is some mystic energy or force that affects your health and can become out of balance. That’s nonsense, not science, but by coupling that nonsense with science, naturopaths came make it look like they are onto something (just as I did with electroism).

So, what do naturopathic treatments actually involve? Well, quite a few things. There is generally a large focus on eating better, exercising more, and making lifestyle changes. Again, this is all well and good, and actual doctors would have no problem with that. If you go to a real doctor with high blood pressure, you will likely get medication, but you’ll also almost certainly be told to eat healthier and exercise more.

Other treatments are more problematic. They include things like homeopathy. I’ve written about homeopathy before, but in short, it is based on three fundamental principles, all of which are ridiculous. First, it argues that “like cures like.” This means that something that causes a set of symptoms in a healthy person will cure an ailment that causes the same symptoms. For example, some homeopathic sleep aids use caffeine as the active ingredient, because caffeine causes the symptom of sleeplessness in a healthy person (see why I said this was ridiculous?). Not to worry though, homeopaths “solve” this by invoking their second principle: dilution. According to homeopathy, diluting something will make the beneficial properties stronger while removing the harmful properties. If that sounded like utter nonsense, that’s because it was. That’s clearly not how dilutions work. Further, homeopathic remedies are often literally so dilute that not a single molecule of the active ingredient remains. In other words, they are literally just water. This is where the third principle comes in. According to homeopathy, water has memory, but somehow it only remembers the beneficial properties of the long-gone active ingredient while forgetting all of the poo, dirt, and other things it has come into contact with. Homeopathy is utterly absurd. It is hokum that is completely at odds with science, reality, and common sense, yet it is often a part of naturopathy.

Naturopathy also often includes various components of “traditional Chinese medicine” such as acupuncture (though this particular review excluded TCM). I’ve written about acupuncture at length before, but briefly, there is no known mechanism, accupoints don’t actually exist, and studies on it have found very inconsistent results, with even the positive studies reporting only extremely slight improvements. This strongly suggests that it is just a placebo.

Another core “principle of practice” in naturopath is the “healing power of nature.” In other words, naturopathy (as its name suggests) is built on the notion that “natural” treatments are automatically better than “artificial” ones. This is not a scientific concept. In fact, it is a logical fallacy known as an appeal to nature. The fact that something is natural tells you absolutely nothing about whether or not it is safe and beneficial. Nature is not a kind entity that is looking out for your vest interests, Cyanide is, after all, natural. Nevertheless, naturopaths use this principle as the justification for prescribing all manner of herbs, supplements, and other “natural remedies.” The science behind these varies, but as a general rule, they either haven’t been tested or have failed testing, otherwise, they would be a part of mainstream medicine (e.g., Aspirin).

There are countless other quack remedies used by naturopaths, but I digress. I realize that this all seemed like a very round about way to introduce the study, but it was important groundwork for what the authors did. You see, when things like homeopathy, acupuncture, and most herbal remedies undergo proper, controlled scientific testing, they tend to fail those tests. To solve this, naturopaths (and the authors of this study) argue that we have to look at naturopathy as a whole, rather than studying the particular methods it uses. So, the authors ignored all of the studies looking at individual components of naturopathy and only included studies that included multiple “modalities” (treatments), the vast majority of which included diet and exercise as some of the treatments!

I want you to note that this is exactly the flawed reasoning that I used to support electroism in my example, and it fails for all the same reasons. Which actually makes more sense, that homeopathy only works when coupled with diet and exercise or that diet and exercise work and homeopathy is pseudoscientific bunk that was just along for the ride? This review guaranteed that naturopathy would look effective by using studies where evidence-based practices (diet and exercise) were included alongside nonsense (just as I could do, in my hypothetical example, by only looking at studies that included diet and exercise alongside a jolt from a battery).

Indeed, when you look at the studies in this review, you have things like a study that took patients with blood pressure problems and prescribed them better diets, more exercise, and herbs/supplements. Then, like a miracle, the patients’ blood pressure improved. Does that mean that the herbs/supplements worked? Of course not! The supplements are completely confounded by the diet and exercise, which we know work! In other words, because most of the tests included diet and exercise, they were totally confounded and do not let us draw any firm conclusions about the individual aspects of the treatments. All of the benefits could be entirely from the improved diet and exercise. At best, all that this review actually does is reaffirm that diet and exercise are beneficial. No shit, Sherlock.

Just in case my point here isn’t clear, let me try a different example. Good tires are known to improve your car’s fuel mileage. Now, imagine that someone tries to sell you a magic air freshener which will re-vitalize your car and balance its energy, but they tell you that it only works if you also install better tires. That person would obviously be a charlatan, right? The air freshener is irrelevant to the improvements in your car, and if it actually could improve your car, then it should be able to provide at least small improvements on its own. The same is true of naturopathy.

Finally, you may be thinking, “who cares if some of it is bunk if the net effect is improvement?” Responding to this is a whole post in itself, so I will simply say that the rational response is to increase the good parts of naturopathy in actual medicine (i.e., diet, exercise, good doctor-patient relationships) and discard the junk, rather than continuing to fund and perpetuate nonsense. This is especially important since, in many cases, people seek naturopathic remedies instead of actual medical treatment.

My point in all of this is simple. This study, and proponents of naturopathy in general, use the deceptive tactic of combing utter nonsense with exercise and diet, then pointing to improvements as evidence that the utter nonsense works. This strategy is both logically and scientifically flawed. These tests are completely confounded and, therefore, scientifically invalid. To see if things like supplements and homeopathy work, we have to eliminate confounding factors, and when we do that, they generally fail.

Note: there were multiple other problems with the review and the studies it cited, which you can read about at Science-Based Medicine.

Related posts

Advertisements
Posted in Vaccines/Alternative Medicine | Tagged , , , | 13 Comments

Vaccines and autism: A thorough review of the evidence (2019 update)

One of the most common concerns that people have about vaccines is that they might cause (or exacerbate) autism. This idea is perpetuated by celebrities and innumerable websites, and it has become one of the cornerstone arguments of the anti-vaccine movement, but is there any truth to it? Perhaps unsurprisingly, both sides claim a superiority of evidence. Indeed, you can find numerous websites presenting lists of papers that they claim provide evidence that autism is caused by vaccines (such as Ginger Taylor’s list of “157 research papers supporting the vaccine/autism link“). Conversely, those who support vaccines also have lists of papers which they present as evidence that vaccines do not cause autism (for example, here, here, and here).

Which is correct? The internet is full of misinformation on this topic, so I want to cut through that crap and talk about the actual studies themselves rather than simply tossing lists around. In science, quality is often far more important than quantity, and you have to critically examine studies rather than blindly believing them (details here; here and here; examples here and here). So that is what I want to do in this post. I am going to walk you through both the anti-vaccine and pro-vaccine lists to see which position is actually supported by the evidence. I did this once before several years ago, but since then, the anti-vaccine lists have grown from 124 papers to 157 papers, and conversely, two new papers showing that vaccines do not cause autism have been published and are worth talking about. Therefore, I have updated the original post with the new papers on both sides, as well as re-writing some sections for improved clarity (you can read my original post here). You can also find excellent debunkings of the anti-vaccine lists at Stories from the trauma bay and I Speak of Dreams (here and here). I highly recommend reading those posts in addition to my own.

I have three key target audiences here. First, to any parents who are concerned about vaccines and are truly and sincerely looking for good information, I hope that this post will be a helpful tool for you and will dispel much of the nonsense on the internet. There are so many frightening stories and claims out there that I fully understand why you would be concerned. So I have done my best to thoroughly cover all of the evidence, and I hope that you will carefully consider it.

Second, for those who have already reviewed the evidence, but are tired of explaining it over and over again in debates, I hope that this post will provide a resource that will save you some time.

Third, for those who are not particularly interested in the autism/vaccine debate, I hope that this post will provide a nice worked example of how to critically analyze a large body of literature. There are, sadly, a lot of bad scientific publications out there, and it is important that you know how to sift through them and separate the high quality studies from the statistical noise.

Finally, to those who are already convinced that vaccines cause autism, although you are not my target audience, I do hope that you will read this, but I have one simple request to make of you. If you choose to continue reading, then I want you to seriously consider the possibility that you might be wrong. I want you to actually examine the evidence presented here rather than blindly ignoring it. If you aren’t willing to do that, then there is really no point in you continuing to read.

This post is necessarily long, so for your convenience, I have included a short summary section at the beginning which it condenses the entire post into a few paragraphs. If you really want to understand this topic, however, I suggest that you skip past this section and read the detailed analyses of both the pro-vaccine and anti-vaccine literature. I have made a list of hyperlinks (below) that you can use to jump to any particular sections that interest you. You can also download an excel sheet with citations for Ginger’s list of anti-vaccine papers along with how I categorized them.

Following popular request, I have thrown together a crude .pdf of this post that you can download here.

Note: Throughout this study, I will refer to “pro-vaccine studies” and “anti-vaccine studies.” I am simply using those terms as an easy way to distinguish between studies that failed to find an association between vaccines and autism and studies that found an association (or are cited by ant-vaccers as evidence). Please do not misconstrue those terms or view them as implications that the research was biased or agenda driven.

About this update
When I wrote the original version of this post in 2016, Ginger’s infamous list of papers only included 124 entries. Since then, it grew to 130 then 157, so it seemed like it was time for an update. I was particularly curious to see what new evidence Ginger had found. In the original version of this post, I had pointed out that the evidence against the autism/vaccine hypothesis was extremely strong, with multiple high quality studies with tens of thousands of children, and even a meta-analysis with over one million children. Thus, for the anti-vaccine position to be viable, it would have to, at the very least, have equivalent evidence.

So what did Ginger come up with? Of the 34 studies she added, an astounding 23 papers weren’t about vaccines, one wasn’t about autism, four claimed to be about autism but were animal studies so their applicability is limited, one was an in vitro study (again, limited applicability), nine were non-systematic reviews or opinion pieces, and ten were on forms of mercury that were never in vaccines or thimerosal (which hasn’t been in childhood vaccines in almost two decades; note: some studies were in multiple categories).

Only two of the new papers were on novel research that was conducted on humans and is relevant to the current vaccine schedule. However, both of those papers were published in a predatory journal (i.e., not a real science journal) and were based on an utterly terrible and completely invalid survey of homeschoolers. Indeed, one of those papers has already been retracted (more on those studies later).

Amusingly, she also added Khaled et al. (2016) to the list. This is entertaining because, while the study made no mention of vaccines, it did identify biomarkers which can be used at birth (i.e., before vaccines) to diagnose autism. In other words, it showed that autism is diagnosable before being vaccinated. Thus, this paper actually provides good evidence against Ginger’s claims (the irony is incredible).

It’s also worth briefly mentioning what Ginger didn’t do with the list. Namely, remove the irrelevant papers. As Liz Ditz, Doc Bastard, and I have been pointing out for years, many of the papers on the list simply have nothing to do with vaccines, autism, or both. Many of them literally don’t even mention vaccines, and the fact that Ginger refuses to remove them (and continues to add more irrelevant studies) shows a shocking level of either ignorance or intellectual dishonesty (most likely both).

To be fair, she did remove Inbar et al. (2016) from the list after that paper was withdrawn from publication, but that doesn’t change the fact the she refuses to critically examine most of the papers on her list and is clearly more interested in having a high paper count than in accurately reporting the state of the literature.

On the other side of things, several new studies have provided more evidence that vaccines don’t cause autism, and there are two that I have added to the discussion in this post. First, I’ve added Hviid et al. (2019), the massive Danish cohort study that has been in the press lately. Second, I added Goin-Kochel et al. (2016), which compared vaccination rates among children with different types of autism to see if there was an association specifically with regressive autism (as many anti-vaccers claim). Beyond the novel studies, I have also added Fombonne et al. (2006; a large time-series study in Canada) and Makela et al. (2002) which looked at the timing of autism diagnosis following vaccination.

Additionally, I have removed Price et al. (2010) and Verstraeten et al. (2003) from my main count of large studies, and I have re-framed how I discussed them. I did this because they specifically examined thimerosal containing vaccines, so removing them from the main count seemed more consistent with how I treated the anti-vaccine studies on thimerosal, and, unlike Ginger, I really do strive for intellectual honesty. I apologize if my previous treatment of those papers caused any confusion. It was not my intention to mislead anyone.

Finally, I have rewritten several parts of this, expanding some sections, shortening others, and even adding some new sections. So this is a fairly major overhaul of the previous version.

hierarchy of evidence

Not all study designs are equal (details here)

TL;DR
I want to start with the pro-vaccine lists (for example, here and here). These lists boast well over 100 studies, but many of those papers are admittedly small, used relatively weak designs, are non-systematic reviews, or aren’t relevant for the current vaccine schedule. So I am just going to focus on the really high quality evidence, because there is a lot of it.

Several studies have either looked for general correlations between autism rates and vaccines (Fombonne et al. 2006; Dales et al. 2001) or have looked for changes in autism rates following either the introduction (Taylor et al. 1999; Chen et al. 2004) or removal (Honda et al. 2005) of the MMR vaccine. None of these studies found a significant relationship.

Other studies have used a more robust design known as a case-control analysis (Destefano et al. 2004 [642 people with autism; 1824 people without autism]; Smeeth et al. 2004 [991 with*; 4469 without]; DeStefano et al. 2013 [256 with; 752 without]; Uno et al. 2015 [189 with, 224 without]). Case-controlled studies start with two groups (one with and one without the effect of interest) then work backwards to test a potential cause. This makes them a very powerful design for detecting associations with relatively rare events (such as autism*), yet none of these studies found those associations.

*Note: Although autism is common, it is still rare enough that you need very large sample sizes for most studies before you can detect significant changes. Case-control studies solve that problem by starting with a group that already has autism, then working backwards.

Next, we have cohort studies that compared autism rates between children who did and did not receive the vaccine being stuided. This is one of the most powerful experimental designs, and these studies were particularly large (Hviid et al. 2019 [657,461 children]; Madsen et al. 2002 [440,654 children]; Anders et al. 2004 [109,863 children]; and Jain et al. 2015 95,727 children]). Take another look at those sample sizes, they are enormous (far larger than any of the anti-vaccine studies), but once again, they did not find any significant differences between vaccinated and unvaccinated children.

Further, there is a meta-analysis with over 1.2 million children (Taylor et al. 2014) which also failed to find evidence of vaccines causing autism. Meta-analyses are the most powerful type of paper because they combine the results from multiple studies, thus greatly reducing the odds of a false result. Further, the larger the sample size, the more powerful the study. So this meta-analysis is the most powerful method available, and it used an incredibly large sample size, which makes it an extremely robust and convincing study.

Additionally, there are also studies that looked at whether vaccines can specifically cause regressive autism, and they also failed to find a significant effect of vaccines (Richler et al. 2006; Uchiyama et al. 2007; Goin-Kochel et al. 2016).

On the anti-vaccine side, I went through their lists of papers (here, here, and here; 160 papers total), and 33 of them weren’t actually about autism, 82 (over half) weren’t about vaccines, 41 were animal trials or in vitro studies (which are weak designs that have limited applicability to humans, especially for something like autism), 60 were on either a form of mercury that has never been in vaccines or thimerosal (which hasn’t been in childhood vaccines for almost two decades), 9 were case reports/conference abstracts/opinion papers/some other non-research paper, and 37 were non-systematic reviews (only 8 of which were relevant to the topic at hand; some papers were in multiple categories).

Only 14 of the papers were actually studies on humans that are relevant to autism and the current vaccine schedule, but none of those studies are large, all of them were association studies (i.e., they could not show causation, because correlation does not equal causation) and most of them were seriously flawed. Additionally, many of them relied on the idea that autism rates are increasing, but there is a substantial amount of evidence that the increase is largely (if not entirely) due to changes in how autism is diagnosed rather than an actual increase in the number of people with autism (Rutter 2005; Taylor 2006; Bishop et al. 2008; Baxter et al. 2015; Hansen et al. 2015). Finally (and most amusingly), one of those 14 studies was actually a pro-vaccine study that directly contradicted the results of two of the other studies.

Thus, when you consider all of the evidence, it is completely fair to say that the scientific evidence overwhelmingly supports that conclusion that vaccines do not cause autism, and there is no reliable evidence to the contrary. To be clear, that’s not just the opinion of a blogger. Rather, at least seven systematic reviews have looked at the literature and come to the exact same conclusion (Jefferson et al. 2003; Klein and Diehl. 2004; Parker er al. 2004; Hurley et al. 2010; Stratton et al. (eds). 2011; Demicheli et al. 2012; Margaret et al. 2014).

To be fair, the anti-vaccers also have reviews, but none of their reviews were systematic (Rimland and McGinnis. 2002; Singh. 2009; Ratajczak 2011; Sienkiewicx et al. 2012;Shaw and Tomljenovic. 2013b; Shaw et al. 2014a; Morris et al. 2017a; Lyons-Weiler 2018). This is important because systematic reviews use pre-defined search terms and criteria to find papers. Thus they include all of the relevant papers, regardless of whether they were pro or anti-vaccine. In contrast, non-systematic reviews include whatever papers the authors felt like including. As a result, it should not surprise you to learn that the anti-vaccine reviews ignored the large meta-analysis, cohort studies, etc. and instead focused on the small studies. In other words, they painted an inaccurate and deceptive picture that did not represent the actual state of knowledge. Further, none of the papers cited in those reviews actually presented concrete evidence of vaccines causing autism. Rather, the reviews constructed hypotheses about how vaccines could in concept cause autism. That would be fine if it wasn’t for the fact that those hypotheses have been tested and discredited via the case-controlled studies, cohort studies, etc. In other words, if the hypotheses were true, those studies should have found evidence of vaccines causing autism, but they didn’t. Therefore, we must reject the hypotheses.

In short, the idea that vaccines cause autism has been extremely thoroughly tested by numerous scientists working for different universities and organizations from around the world. It has been tested via multiple different methods and populations, and it has been addressed from multiple angles (e.g., different vaccines, different vaccine components, age at vaccination, number of antigens, number of doses, etc.), and the result is exceptionally clear: vaccines do not cause autism. There are no large, properly controlled, epidemiological studies that disagree with that result. For more details about both the pro and anti-vaccine studies, please read the rest of this post.

Note: very few of the pro-vaccine papers had conflicts of interest (i.e., they were not funded by pharmaceutical companies), and conflicts of interest were also present in many of the anti-vaccine papers. More details are provided in the rest of this post.

Jump back to the top

The Autism/Vaccine Hypothesis
Science is all about making hypotheses and testing whether or not their predictions come true. Therefore, it is always a good idea to establish exactly what you are asking before you look at the literature. In this case, the question is whether or not vaccines cause autism, but that needs to be nuanced a bit. According to the CDC, about 1 in 59 children in the US have autism, with other developed countries reporting varying, but largely comparable levels (Elsabbagh et al. 2012). Those numbers have gone up over time, which has led anti-vaccers to refer to the situation as an “autism epidemic,” and they often make dire predictions like, by 2022 autism rates will be 1 in 9. I’m not going to take the time to explain why that math is absurd,  but I will point out that there is a large body of evidence showing that most, if not all, of the increase in autism rates is due to changes in how autism is diagnosed (Rutter 2005; Taylor 2006; Bishop et al. 2008; Baxter et al. 2015; Hansen et al. 2015). In other words, autism rates are higher now than they were in 1990 because people who would not have been considered autistic in 1990 are considered autistic today (Dr. Novella wrote a good post on this several years ago that includes some additional sources).

The point is that we have two different hypotheses that make different predictions. If vaccines are actually causing an autism epidemic, then when we compare vaccinated and unvaccinated children, we should find that autism rates are much lower among the unvaccinated children. If vaccines don’t cause autism, however, then the rates should be the same. Importantly, the larger our sample sizes, the more power that we will have to detect significant differences. In other words, even if vaccines are only responsible for a very small portion of autism cases (rather than an epidemic), we could still detect that with a large enough sample size. Now, with that in mind, let’s see what we find in the literature.

Jump back to the top

Studies that failed to find evidence of vaccines causing autism
I want to start with the pro-vaccine studies, and there are quite a few of them. Indeed, you can find several lists on the internet that boast over 100 studies demonstrating that vaccines do not cause autism (for example, here and here). I am, however, a bit cautious about lists like this. They can be quite useful, and I have linked to them multiple times on this blog, but a quick examination of these lists will reveal that they do actually contain quite a few low quality studies with weak designs or tiny sample sizes, opinion papers, etc. So I am not going to talk about all of the studies in these lists. Rather, I have painstakingly gone through them to eliminate all of the studies with really weak designs (like animal studies), tiny sample sizes, questionable statistics, etc. This type of filtering is a really good idea when you are examining a topic because it weeds out the statistical noise and leaves you with the reliable studies (importantly, however, you need to have a good understanding of experimental design, statistics, etc. before you can do this properly).

My filtering left me with six correlation/time-series studies, five case-control studies, four cohort studies, one meta-analysis, and seven systematic reviews that examined the hypothesis that vaccines cause autism. These studies provide the backbone of evidence, but there are many other studies which build on that backbone, some of which I will discuss. Additionally, many of the smaller studies that I left out are uncompelling on their own, but their cumulative effect is convincing (see the section on systematic reviews).

Jump back to the top

Correlation/time-series
Several different designs were used for these studies. One of them simply looked for correlations between vaccine coverage and autism rates (Dales et al. 2001). If vaccines cause autism, we would, of course, expect there to be higher autism rates when vaccine coverage is higher; however, this study failed to find that trend (i.e., vaccination rates and autism rates were not correlated). Similarly, Fombonne et al. (2006), used data on 27,749 children in Montreal, Canada from 1987 to 1998 to look at pervasive developmental disorders (including autism) over time and how they associated with MMR vaccine coverage, MMR dose (one vs two), and thimerosal exposure. There were no associations in any comparison, and total developmental disorders were actually highest when MMR coverage was the lowest

Another pair of studies specifically looked for changes in autism rates when the MMR vaccine was introduced to the UK (Taylor et al. 1999; Chen et al. 2004). The MMR vaccine is a favorite target of anti-vaccers, and if it actually causes autism, then we should see a spike in autism rates immediately following the introduction of that vaccine into a population, but neither study detected a significant change.

Another study, (Honda et al. 2005) took the opposite approach. In 1993, Japan abruptly stopped using the MMR vaccine, so Honda et al. (2005) examined the autism rates across the entire city of Yokohama (roughly 300,000 people) from 1988-1996. This study provides a nice balance to  Taylor et al. (1999) and Chen et al. (2004), because just as you would expect autism rates to spike following the introduction of MMR (if MMR caused autism), you would also expect the rates to drop after the vaccine is removed. Just like the previous studies, however,  Honda et al. (2005) failed to find any evidence of the vaccine causing autism (i.e., autism rates did not drop when the MMR vaccine was removed).

It’s worth pointing out that these types of studies cannot establish causation, because they only show correlation, and correlation does not automatically equal causation. The fact that two things occur together does not mean or even suggest that they are causally related. Nevertheless, a lack of correlation does suggest a lack of causation. In other words, if X causes Y, then X will be correlated with Y. So if X and Y are not correlated, that also suggests that there is no causal relationship between them.

Finally, Makela et al. (2002) took a very different approach and took a group of 535,544 who received the MMR vaccine and followed them to see if and when they developed autism. The idea was that if the anecdotes about vaccines quickly leading to autism were correct, then autism diagnoses should have clustered shortly after receiving the vaccine. As you might have guessed, they did not cluster (I talked about why we expect some cases to follow vaccines just by chance here).

Note: this type of time-series design was not included in the original image on the hierarchy of evidence that I have been including throughout this post, but these studies would usually rank about the same as a cross sectional study.

Jump back to the top

Case-control studies can't establishes causation, but they are one of the best designs for looking for associations with rare relatively rare outcomes (e.g., autism). Details here.

Case-control studies can’t establishes causation, but they are one of the best designs for looking for associations with rare relatively rare outcomes (e.g., autism). Details here.

Case-control studies
Case control studies are another type of correlation study, but rather than simply looking for correlations in a population, they take two groups that are similar except for an outcome (e.g., autism). They then work backwards to test for a potential cause of that outcome (e.g., you compare a group with and without autism to see if they differ in their vaccination rates). This design can only show association rather than causation, but it is very a very powerful way of at looking at relatively rare outcomes, and it allows you to get very robust results out of relatively small sample sizes (compared to other designs), making it an excellent method for looking for associations between vaccines and autism.

I found four reasonably large case-control studies that examined autism rates and are relevant to the current vaccine schedule (Destefano et al. 2004 [642 people with autism; 1824 people without autism]; Smeeth et al. 2004 [991 with*; 4469 without]; DeStefano et al. 2013 [256 with; 752 without]; Uno et al. 2015 [189 with, 224 without]). None of these studies found evidence that vaccination was associated with the development of autism.

Additionally, although all of these studies addressed the question of vaccines and autism, several of them were focused on a particular facet of the question or examined multiple sub-questions. For example, Destefano et al. (2004), Smeeth et al. (2004), and Uno et al. (2015) looked specifically at the MMR vaccine, whereas DeStefano et al. (2013) took an entirely different approach and looked at antigen exposure. This is really important because one of the most common tropes of the anti-vaccine argument is the, “too many too soon” argument, which argues that the antigens in vaccines will overwhelm a child’s immune system and lead to problems like autism. This study directly addressed that concern.

Several of these studies also looked at whether or not the age at vaccination was important for the development of autism, thus addressing the argument that vaccines should be delayed (Destefano et al. 2004; Smeeth et al. 2004; Uno et al. 2015). In all fairness, Destefano et al. (2004) did find slightly higher vaccination rates among the autistic children for their 36 month age-group (93.4% vs 90.6%), but there were no differences at 18 or 24 months, and in the case of the 36 month-olds, many of them had started to show signs of autism before receiving the vaccine, so the vaccine was clearly not at fault.

Another study (Goin-Kochel et al. 2016) took a different approach. Many anti-vaccers have shifted the goal posts from saying that vaccines cause autism to saying that vaccines specifically cause regressive autism. They use this to try to dismiss many of the existing studies by arguing that they looked at autism generally, rather than specifically looking at regressive autism. This argument is a logically invalid ad hoc fallacy (i.e., it is an assumption being made to patch a hole in an argument), but Goin-Kochel et al. (2016) tested it anyway. They took a large group of people with autism, grouped them by type of autism, then compared vaccination rates among the groups. If vaccines specifically cause regressive autism (as anti-vaccers suggest), then the vaccination rates should have been higher in the regressive autism group, but the study found no consistent significant differences among groups. This conclusion has also been supported by other studies (Richler et al. 2006; Uchiyama et al. 2007).

Finally, I want to briefly mention Price et al. (2010). This case control study (256 with autism; 752 without) looked specifically at thimerosal exposure from vaccines and immunoglobulin injections. It also found no associations. I have not included it in my main count of studies, however, because thimerosal was removed from childhood vaccines nearly two decades ago (the exact date varies by country, but its usually 1999-2001; certain formualtions of the flu vaccine still have thimerosal, and in Austrlaia, the Hep B vaccine still has it). Thus, thimerosal studies are no longer relevant for the current vaccine schedule.

*Note: Smeeth et al. (2004) also did a larger analysis of 1294 people diagnosed with “pervasive developmental disorders” which included those diagnosed with autism and those diagnosed with other developmental disorders, and that analysis also failed to find a significant effect of vaccines.

Jump back to the top

Case-controlled studies can be very powerful (especially with large sample sizes) and can actually show causation (details here).

Cohort studies can be very powerful (especially with large sample sizes) and can actually show causation (details here).

Cohort studies
Next, we have the cohort studies. These work in the opposite direction from case-controlled studies. They start with a group of people (cohort) that are similar except for their exposure to some potential cause (e.g., vaccines). Then, the researchers track these individuals to see if they eventually differ in some outcome (e.g., autism). In other words, if vaccines cause autism, then you expect the members of the cohort that received vaccines to develop autism significantly more frequently than the members of the cohort that were unvaccinated.

I found many small cohort studies, but only four of them were large enough to be worth talking about in the context of the modern vaccine schedule (IMO). Those four studies were, however, extremely large and provide very convincing evidence that vaccines do not cause autism. Anders et al. (2004) used 109,863 children to study the DTP/DT vaccine, Hviid et al. 2019, Madsen et al. (2002) and Jain et al. (2015) used 657,461 children, 440,654 children, and 95,727 children (respectively) to examine the MMR vaccine. Really think about those numbers for a minute. Those sample sizes are extraordinary and gave the researchers tremendous power to detect significant trends, yet none were found.

As with the case-control studies, several of studies also examined additional aspects of the topic. For example, Madsen et al. (2002) also examined the effects of age at vaccination, and did not find a significant effect. Hviid et al. (2019) examined several other factors, such as the timing of an autism diagnosis following a vaccination (like Makela et al. 2002, which I discussed earlier). They also tried grouping children by previous vaccination history (i.e., vaccines other than the MMR) to see if the cumulative effect of lots of vaccines plus the MMR would make a difference, and the results were still negative. This is a very useful result because anti-vaccers often criticize these studies for only looking at a single vaccine, but this study included vaccine history in its analyses and still failed to find evidence of vaccines causing autism.

Perhaps most importantly, Jain et al. (2015) also looked specifically at a subset of 1,929 children who had a sibling with autism. This is a brilliant design because we know that autism has a genetic component. Even most anti-vaccers agree with that, they just argue that the genes make you more susceptible to the effects of vaccines. If that was the case, however, then it should be much easier to detect vaccine-associated autism in children who have a sibling with autism. In other words, infants who have an older sibling with autism have a higher risk of developing autism than infants whose siblings do not have autism (i.e., if your sibling has autism, then it is likely that the genes that predispose you to it are in your family). This means that by examining siblings, you are looking at a “high risk” group, thus maximizing your statistical power. Even with this design, however, they failed to find any significant effects of vaccines.

Hviid et al. (2019) also incorporated this sibling design and failed to find a significant effect, but their sample size for this subset test (37 children) was admittedly too small to be compelling. However, they also used other factors that are thought to increase a child’s risk of autism to create subgroups of children with a moderate risk (2312 children) or high risk (1048 children), then the looked for associations between autism and vaccination in those subgroups. Once again, there were no significant effects of vaccines. This lack of association in high risk groups is very strong evidence that vaccines are not a cause of autism.

Finally, I want to briefly talk about Hviid et al. (2003) and and Verstraeten et al. (2003). I did not include those studies in my main list because unlike all of the other studies that I have been talking about, they looked specifically at thimerosal (which is no longer in childhood vaccines). Verstraeten et al. (2003) used a cohort of 124,170 to look at cumulative thimerosal exposure, while Hviid et al. (2003) use a cohort of 446,695 children to compare vaccines with thimerosal to vaccines without thimersoal. Discussions of thimerosal are fairly pointless since it was removed from childhood vaccines almost two decades ago. Nevertheless, I want to make a few brief points.

First, anti-vaccers often accuse thimerosal of being the causative agent of autism (despite its absence from the current schedule), and these massive studies provide powerful evidence against that hypothesis. Second, Hviid et al. (2003) also looked at dose effects and failed to find any evidence of increasing autism with increasing vaccine dose. This is significant, because even if there was something in vaccines other than thimerosal that was causing autism, you would still expect that to show up in this study. Finally, the inclusion of dose once again provides evidence against the “too many too soon” argument.

Jump back to the top

Systematic reviews and meta-analysis are the highest category of evidence because they combine the results of multiple studies, which makes them less prone to false conclusions (details here).

Systematic reviews and meta-analysis are the highest category of evidence because they combine the results of multiple studies, which makes them less prone to false conclusions (details here).

Systematic reviews
At the start of this section, I want to distinguish between a systematic review and a general or non-systematic review. A systematic review looks for papers using a predefined set of search terms and databases. For example, you might search PubMed and Web of Science for papers containing both the terms “vaccine” and “autism.” Then, you take that list of papers and filter by some other pre-defined criteria. For example, you might be interested only in studies that were case-controlled, cohort, or randomized controlled trials. Then, you write the review on the papers that are left.

Setting up reviews this way with pre-defined search terms and inclusion criteria is a really powerful way to comb through the scientific literature because it avoids bias. If a study meets your criteria, then it gets included regardless of what its conclusions were. Thus, you get a fair and accurate representation of the literature. To be clear, you can still bias these if you use absurd search terms (like, “vaccines do not cause autism,” for example), so you should always check systematic reviews to see what their inclusion criteria were.

In contrast, non-systematic reviews use any papers that the authors thought should be included. These reviews can still be quite good, but they also can be very problematic because the authors’ biases can have major influences over the papers that get included. If you are anti-vaccine, for example, you can write a “review” that only includes anti-vaccine papers, and ignores all of the pro-vaccine papers, thus creating the illusion that there is an overwhelming amount of evidence against vaccines (we’ll encounter some of those later in this post). The inverse is, of course, also true. A pro-vaccine scientist can bias a review just as easily as an anti-vaccine scientist. This is why systematic reviews are much better than regular reviews, and when you have multiple systematic reviews that all agree with each other, you can be fairly confident that the literature is pointing in a consistent direction.

In the case of autism, that direction is most decidedly away from a link between autism and vaccines. I found seven different systematic reviews of the topic, and they all said the same thing: the evidence does not support an association between vaccines and autism (Jefferson et al. 2003; Klein and Diehl. 2004; Parker er al. 2004; Hurley et al. 2010; Stratton et al. (eds). 2011; Demicheli et al. 2012; Margaret et al. 2014). I also found several non-systematic reviews that concluded that vaccines don’t cause autism, but I decided not to include them for the reasons explained above.

On a side note, Demicheli et al. (2012; a lengthy and thorough Cochran review ) also found that, “Exposure to the MMR vaccine was unlikely to be associated with autism, asthma, leukaemia, hay fever, type 1 diabetes, gait disturbance, Crohn’s disease, demyelinating diseases, bacterial or viral infections.”

Jump back to the top

Meta-analysis
Finally, we have a meta-analysis. Meta-analyses are the single most powerful tool available to scientists, because they actually pool the results of multiple studies, and run statistics on that pooled data set. This provides much larger sample sizes than you could normally achieve, and it largely overcomes the fact that sometimes a study reaches the wrong conclusion just by chance (i.e., the odds of the pooled data from multiple studies producing an erroneous conclusion is much, much lower than the odds of a single study being wrong). As a result, these studies are considered to be the highest level of evidence.

There is only one large meta-analysis for vaccines and autism, but it’s a big one (Taylor et al. 2014). It had a sample size of over 1.2 million children, which is an extraordinarily large sample size that provides tremendous statistical power. Nevertheless, this study did not find any associations between autism and vaccines (or thimerosal or mercury for that matter). That is as conclusive of an answer as you could ever hope to have.

Jump back to the top

Randomized-controlled trials
There are no large randomized controlled trials on vaccines and autism, and there are two very good reasons for that. First, autism is rare enough that you would need an absurd sample size to have a reasonable chance of detecting a significant effect. So they aren’t practical (case-controlled and cohort studies are more well-suited to the question at hand). Second, the benefits of vaccines have been established beyond any shadow of a doubt, so it would be unethical to deliberately give people placebos rather than vaccines.

The lack of randomized controlled trials is not a problem, however. Randomized controlled trials are the most powerful experimental tool for establishing causation, but the other methods are perfectly capable of showing a lack of causation. For example, case-controlled trials can only show correlation, not causation, but since a lack of correlation also means a lack of causation, they can be very powerful tools for showing that two things are not causally related. So if you were arguing that vaccines cause autism, then a lack of randomized controlled trials would be potentially problematic (depending on the strength of the other studies, especially the cohort studies), but the lack of randomized controlled trials is really irrelevant for this negative result.

Jump back to the top

Regressive autism, multiple doses, age at vaccination and other sub-hypotheses
At this point, I want to pause for a minute to deal with some common criticisms and point out just how broad the literature is. Anti-vaccers are masters at shifting goal posts, and, as a result, scientists are forced to study a constantly shifting hypothesis. It’s like fighting the hydra. Every time we falsify one hypotheses, anti-vaccers invent two more, rather than simply accepting the results.

For example, a common criticism I hear is that many of the studies I’ve been talking about focused specifically on the MMR vaccine. However, this criticism ignores the fact that scientists have focused on the MMR vaccine because that is the vaccine that anti-vaccers singled out as the cause of autism! In other words, scientists tested anti-vaccers’ hypothesis, but anti-vaccers didn’t like the answer to those tests, so they changed the hypothesis, then accused scientists of not addressing the “real issue” (also, there have been large studies that looked at vaccines other than the MMR, e.g., Anders et al. (2004)). I’ve tried to point out some of these hypotheses as I’ve gone through the studies, but to make it easy, here is a non-exhaustive list of sub-hypotheses that scientists have looked and falsified (specifically within the overarching hypothesis that vaccines cause autism):

I also want to point out that some of the more popular hypotheses that have not been directly addressed have actually been discredited by the existing literature. For example, the hypothesis that aluminum in vaccines is the culprit (which became popular once it was clear that thimerosal wasn’t the problem) should have caused an increase in autism among vaccinated children in studies that looked at things like dose and age at vaccination. The fact that those studies were negative discredits the hypothesis.


Jump back to the top

Conflicts of interest
At this point, you may be thinking, “but all of those studies were conducted/paid for by pharmaceutical companies, so they can’t be trusted!” I anticipated that argument and checked for conflicts of interest as I went through these papers. I only found eight, and only two of them were serious enough to be at all concerning.

First, for two of the case-controlled studies (Smeeth et al. 2004; Price et al. 2010), some of the authors had previously received funding from pharmaceutical companies for other projects. In other words, these two studies were not funded by pharmaceutical companies, and the authors don’t work for those companies, but the companies had funded some other projects that were conducted by some of the researchers. That’s hardly damning evidence of corruption.

Two of the other conflicts surrounded Dr. Jefferson. In 1999, he worked as “an ad hoc consultant for a legal team advising MMR manufacturers.” This was acknowledged in his review (Jefferson et al. 2003) and in Demicheli et al. (2012). Jefferson was not an author on the final paper for Demicheli et al. (2012), but the paper stated, “The review authors wish to acknowledge Tom Jefferson and Deirdre Price as previous author.” In other words, he was involved at some stage, but not with the final product. Once again, showing that one of the researchers involved used to be a consultant for a legal team hardly indicates that all of his research has been corrupted. Similarly, Dr Fombonne, who wrote Fombonne et al. (2006), has given advice to pharmaceutical companies, has been on government panels, and has been an expert witness in vaccine lawsuits, but has never received research funding from pharmaceutical companies.

Another one of the conflicts also comes from a review. Hurley et al. 2010 stated, “All authors are employed by MED Communications, Inc., which provides medical and drug information services to multiple pharmaceutical firms, including several manufacturers of various vaccines.” So they were employed by a company that provides info to vaccine manufacturers, but they were not employed by the vaccine manufactures themselves. Again, that’s not really concerning.

Finally, the two potentially serious conflicts are from the funding for Hviid et al. (2019) and Makela et al. (2002). Hviid et al. was partially funded by the Novo Nordisk Foundation. This foundation funds a large number of research projects, but it also owns several pharmaceutical companies (the rest of the funding came from the Danish Ministry of Health). However, as the paper says, “The funder of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report.” The conflict in Makela et al. was more direct. It was partially funded by Merck.

To be clear, conflicts of interest do not automatically negate papers. Conflicts of interest should make you scrutinize a paper more closely, not automatically reject it.

Nevertheless, let’s say that you did want to completely reject those eight studies. That would still leave you with four correlation/time-series studies, three case-control studies, three cohort studies, four systematic reviews, and one massive meta-analysis, all of which provide compelling evidence that vaccines are safe and, as far as I can tell, did not have conflicts of interest. They were funded by numerous different agencies, and the researchers worked for various universities, hospitals, and government agencies from all over the world (you can find more details about the funding for some of these studies here).

I’ll also briefly add here that I have never received any money from pharmaceutical companies, and I do not get paid to write this blog. In fact, it costs me money, because I am using a free wordpress account (so I don’t make money from adds, nor do I have any control over them), and I pay an annual fee for a domain name.

Jump back to the top

Sample size and statistical power
Before I leave this section, I want to make a very important point about that nature of negative results in science. Technically speaking, it is not actually possible to demonstrate that vaccines don’t cause autism, but it is possible to demonstrate that if they cause it, they do so at a very low rate. In other words, if you have a sample size of 100 people, you haven’t tested the possibility that vaccines cause it at a rate of 1 in 1000, and if you have 1000 people, you haven’t tested the possibility that vaccines cause it at a rate of 1 in 10,000, etc. (note: those numbers are not precise). So no matter how large your sample size is, it is always possible that the effect is just smaller than what you were able to detect.

Now, when we apply that to vaccines and autism, what we see is that if vaccines do cause autism they do so at an absurdly low rate. Scientists have conducted a meta-analysis with over 1.2 million children, several cohort studies with close to or over 100,000 children, several large case-controlled studies, and studies that looked specifically at children who were at a high risk of developing autism. This means that we could have detected even a very a very low rate of vaccines causing autism. It’s a bit difficult to calculate exactly what that rate is, but even a rate of about 1 in 10,000 would likely show up in the larger studies.

Remember, however, that the claim being made by anti-vaccers is that vaccines are causing an “autism epidemic,” and that claim is clearly false. Even if vaccines were only responsible for 5% of autism cases, that would be a rate of less than 1 in 1,300 people, which is well within the range that these studies had the power to detect. Further, even if vaccines only caused 1% of autism cases, that would be a rate of 1 in 6,800 people, which is still low enough that we should have detected it.

My point here is that these studies had an extraordinary power to detect even very tiny effects of vaccines, yet they failed to find any evidence of vaccines causing autism. This means, that if vaccines do cause autism, they do so at an incredibly low rate that shouldn’t be concerning (remember, every decision has risks, including the decision not to vaccinate).

To be 100% clear here. I am not in any way shape or form suggesting that vaccines actually do cause autism at a very low rate. There is utterly no evidence to think that they do (thus assuming that they do is an argument from ignorance fallacy). Rather, I am bringing this up because I am trying my best to give a fair and honest representation of the current state of our knowledge, and it is not technically correct to say that we have demonstrated that vaccines don’t cause autism, because what we have actually done is demonstrate that it is very unlikely that vaccines cause autism at a meaningful rate, or, to put it another way, if they cause autism, they do so at an extremely low rate, which still means that anti-vaccers’ claims are false.

Jump back to the top

 

Anti-Vaccers’ Lists of Papers
Introduction to anti-vaccers’ papers
Before I go into the lists of papers that, according to the anti-vaccers, demonstrate that vaccines cause autism, I want to specify exactly what we would need to find in order for anti-vaccers’ claims to be legitimate. Remember, sample size and experimental design are extremely important in determine the quality of a study. Therefore, given that we have very large case-controlled and cohort studies, as well as a meta-analysis with over 1.2 million children, all of which say that vaccines don’t cause autism, for the claim that vaccines do cause autism to be plausible, we would need multiple studies of similar size and power. That’s how science works. You don’t get to trump a massive meta-analysis with a tiny association study. So with that in mind, let’s look at anti-vaccers’ lists of papers, and see if there are any large, high quality studies.

The largest list (to my knowledge) was compiled by Ginger Taylor and currently contains 157 papers “supporting the vaccine/autism link” (previous versions of the list have contained 91, 124, and 130 papers). There’s also, “30 solid scientific studies that prove vaccines cause autism” and “22 medical studies that show vaccines can cause autism.” All three of these lists have been widely shared so you may find them in different places with different names (as well as older versions with fewer studies), but in my searches, these were the three that I found over and over again. For the most part, they are redundant with each other, but I still painstaking went through them one paper at a time to make sure that I hadn’t missed anything. This produced a total of 160 papers.

To get that number down to something manageable, I did a series of filtering steps to get rid of the papers that weren’t worth talking about (you can download an Excel file showing how I categorized them here). Now, before you accuse me of cherry-picking, please remember that I did the same thing for the pro-vaccine lists. I took lists of over 100 papers, and I wilted them down to just the ones that were worth talking about. Further, as you’ll see in a minute, I was far more generous to the anti-vaccine lists than I was to the pro-vaccine lists. For the pro-vaccine list, I filtered by content, study design, sample size, and study quality. In contrast, for the anti-vaccine lists, I only filtered by content and study design, because if I had filtered by the same sample size and quality standards that I applied to the pro-vaccine lists, I would have had exactly 0 studies left to talk about. In other words, none of the studies on those lists are large, well-conducted studies that used a powerful design and provide good evidence that the current vaccine schedule causes autism.

Note: some of the papers that I am not going to talk about were perfectly fine studies, they just have no relevance to the topic at hand. Many of them were, however, seriously flawed. You can find more details about these studies at I Speak of Dreams. and Stories from the trauma bay.

Note: many papers fit into multiple categories, so if you simply try to add up the numbers from each of the following categories, it is going to look like I suck at math.


Jump back to the top

Studies that weren’t about vaccines or weren’t about autism
It may or may not surprise you to learn that an enormous portion of the papers in these lists had absolutely nothing to do with vaccines and autism. Of the 160 papers in the lists, there were 33 that were not about autism and 82 that were not about vaccines! I’m really curious about how a study that never even uses the words “autism” or “vaccine” can prove that vaccines cause autism. To be clear, I was generous in assigning papers to those categories. I did not score animal studies in that category (even though demonstrating autism in animals is essentially impossible) and I didn’t include papers on thimerosal (even though thimerosal is no longer in childhood vaccines). So right off the bat, we have a really good reason to be skeptical of these lists. They were clearly made by people who are either ignorant or deceptive (or both), because they are full of irrelevant papers.

Just to be clear here, when I say that these papers are irrelevant, for the most part, I mean really well and truly, 100% irrelevant. Some of these papers were so off topic that I have absolutely no clue why they were included. For example, Guy et al. (2015) was on the relationship between pre-term birth and autism, and presented evidence that infants who are born prematurely have a higher risk of developing autism. I have absolutely no clue why Ginger thought that paper was evidence that vaccines cause autism (since it’s not about vaccines), and there were lots of papers like that in the lists. For example, Carvalho et al. 2011 was on methods of treating mercury poisoning. How on earth does a paper on methods for treating mercury poisoning show that vaccines cause autism?

Perhaps most amusingly of all, Ginger’s list includes a few gems like Khaled et al. (2016). This was not about vaccines and never mentioned them. Rather, it looked for biomarkers of autism that were present at birth, and it found them! This study clearly doesn’t suggest that vaccines cause atuism and, in fact, it suggests the opposite. If biomarkers can be used to detect autism at birth (i.e., before vaccines) then clearly vaccines aren’t the problem! I think it is pretty cleary that Ginger doesn’t undedrstand the scientific literature.

Other papers were on extremely tangentially related issues. For example, there were several studies on methyl-mercury (e.g., Rice. 1989; Charleston et al. 1994), but methyl-mercury has never been in vaccines. The mercury that used to be in vaccines was ethyl-mercury, which is a very different chemical with different properties. You can’t say, “methyl-mercury does X, therefore ethyl-mercury also does X.” That’s not how chemistry works (also most of those mercury studies weren’t about autism either). Similarly, there were a few papers showing that some component of vaccines were toxic in very high doses, but the dose makes the poison. The fact that chemical Y is dangerous at a very high level is irrelevant to whether or not it is dangerous at the low levels present in vaccines.

Others were about general neurotoxic effects or effects other than autism. Many of these had specific problems that I won’t go into here, but the point that I want to make is that you can’t jump from, “vaccines cause X” to “vaccines cause autism.” Even if these studies had successfully demonstrated that vaccines cause neurological problems other than autism, that wouldn’t indicate or even suggest that vaccines actually cause autism. Each disorder is different with its own set of causes and triggers, and you can’t just assume that the same causal relationships exist for all of them.

The remainder of the studies were, I think, intended to demonstrate various components of hypothetical pathways that supposedly lead from vaccines to autism. For example, there were numerous papers showing associations between oxidative stress and autism, and the argument (I assume) is that vaccines cause oxidative stress, and therefore vaccines can cause autism. There are several problems with that approach, however.

First, the papers on associations between autism and factors other than vaccines were all over the map. They didn’t paint a consistent picture that points towards vaccines. There were studies on associations with the microbiome, associations with oxidative stress, associations with automimue disorders, associations with febrile seizures, associations with mitochondrial disorders, etc. Indeed, Ginger seems to have deluded herself into thinking that any paper on any association with autism is somehow evidence that vaccines cause autism. The list seriously looks like she googled, “causes of autism” then included any studies she found regardless of what it showed.

Second, correlation isn’t causation, and none of those papers established causation, so it could, for example, be the case that some aspect of autism causes oxidative stress rather than the other way around (further, the evidence for vaccines causing oxidative stress is extremely weak).

To put this another way, what the anti-vaccers are doing is laying out a hypothetical pathway in which, for example, they argue that vaccines cause oxidative stress which in turn causes autism, but we don’t know if either of those steps are actually true. Further, even if each step was true independently, you couldn’t actually assume that one will lead to the other in actual patients. The human body is remarkably complex, and biochemical pathways and interactions are complicated and difficult to predict. So it’s often the case that under the right circumstances, A causes B, and under the right circumstances B causes C, but that doesn’t necessarily mean that A will go all the way to C in an actual person.

Finally, and most importantly, in science, you use hypothetical pathways to design experiments, but the pathway itself is not evidence. In other words, you do large studies to see whether or not the pathway is true, and, in the case of vaccines, if this hypothetical pathway actually worked, then the large case-controlled and cohort studies should have found a significant difference between vaccinated and unvaccianted children, but they didn’t. You absolutely cannot present a hypothesis as evidence for your position, but that is exactly what is happening here. These hypotheses have been thoroughly refuted by the large studies that I discussed earlier, and, therefore, they must be rejected.

Jump back to the top

In vitro trials and animal studies
The anti-vaccine lists contained 24 animal studies and 17 in vitro studies (in vitro studies are done on cells and tissues in a petri dish; one study used both methods). There are two important reasons why I am filtering these out (remember that I also eliminated animal trials and in vitro trials from the pro-vaccine lists).

First, and most importantly, these types of studies are always at the bottom of the evidence pyramid. The human body is obviously vastly more complicated than a dish of cells, and in the human body, there are far more chemicals for a drug to interact with, not to mention the fact that you have the kidneys and liver which are actively filtering harmful compounds from your body. As a result, it is very often the case that a chemical will behave one way when directly exposed to a plate of cells (such as nerve cells), but behave very differently in the body where it must travel to the cells without being filtered out or becoming overly diluted, avoid interacting with other chemicals, etc. Similarly, humans, mice, monkeys, etc. are all biochemically different, and chemicals don’t always behave the same way in each species.

Because of all of these factors, in vitro studies and animal trials are preliminary studies that are used as a first pass filtering mechanism. In other words, they are used to decide which topics merit further research, and you absolutely cannot use them as evidence against large epidemiological trials. When an animal trial says X and a cohort study says Y, you go with the cohort study (unless, of course, you can actually find real problems with the cohort study).

The second reason that I am filtering out these trials is that they have very limited applicability to the topic of autism. You cannot diagnose a petri dish as having autism. The best you could do is say, “after being exposed to chemical X, the cells had characteristics that were similar to those of an autistic patient,” but again, association is not causation. So unless you know that those cellular characteristics cause autism, you can’t really say much. Further, showing that a chemical damages a nerve cell does not mean that it will specifically cause autism. Similarly, how would you diagnose autism in a rat? You can say, “the rat is behaving differently,” but again that doesn’t necessarily mean that it specifically has autism.

Finally, I do want to make a few brief comments about a particular set of monkey trials. Anti-vaccers are very fond of citing Hewitson et al. (2008) and Hewitson et al. (2010), which claimed to find evidence of vaccines causing neurological damage in rhesus macaques (anti-vaccers also sometimes cite Hewitson et al. 2009, but that study was withdrawn). All of these were preliminary studies based on ongoing research. Hewitson et al. 2010 literally has the words “pilot study” in the title, and Hewitson et al. 2008 was a conference abstract, not a peer-reviewed paper. Further, the sample sizes were laughably small.

The full study with a more proper sample size has finally come out, and, as often happens in science, the preliminary data were wrong. In the full study, there was no evidence of vaccines causing neurological problems (Gadad et al. 2015). Also, it is worth mentioning that Hewitson is an author on the Gadad paper, and, amusingly, the study was actually funded by an anti-vaccine group!

Jump back to the top

Mercury and thimerosal studies
Mercury is by far the most common vaccine component to be accused of causing autism. Therefore, it should not be surprising that 60 of the papers were specifically about it. However, I’m am not going to talk much about those papers for several reasons.

First, many of these papers were on elemental mercury (Hg) or methyl-mercury, but the preservative used in some vaccines (thimerosal) is actually ethyl-mercury. The mercury in thimerosal does not behave like other types of mercury because it is bound to a ethyl group (just like the chlorine in table salt does not behave like chlorine because it is bound to sodium; details here).

Second, even for the studies on thimerosal, none of them were large human trials that were capable of establishing causation. There were lots of small trials, animal trials, in vitro trials, association studies, etc., but large cohort studies were completely lacking. In contrast, remember that several of the papers in the pro-vaccine list specifically looked at thimerosal (as well as the whole vaccine) and failed to find any relationship. These pro-vaccine studies included a meta-analysis with 1.2 million children  (Taylor et al. 2014), a cohort study with 446,695  children (Hviid et al. 2003), another cohort study with 124,170 children (Verstraeten et al. 2003), and a case-controlled trial with over 1,000 children (Price et al. 2010; several of the other studies also used vaccines that contained thimerosal, but they did not explicitly test that component). These massive, high quality studies completely obliterate the small low quality studies presented by the anti-vaccers. That is how the hierarchy of evidence works.

Finally, and perhaps most importantly, most industrialized countries do not currently have thimerosal in their childhood vaccines, and thimerosal has been absent for nearly 20 years (depending on the country)! In the USA, Canada, and countries in the EU, for example, currently only certain strains of the flu vaccine contain thimerosal. Similarly, in Australia it is absent from all childhood vaccines except for certain Hep B vaccines.

This removal of thimerosal is important for two important reasons. First, if you live in a developed country (which is where most of my readers are) then thimerosal in childhood vaccines is a non-issue for you. Even if early exposure to thimerosal did cause autism (which it doesn’t), thimerosal isn’t in childhood vaccines, so you have nothing to worry about.

Second, the nearly world-wide removal of thimerosal from vaccines provides an excellent test of the hypothesis. If thimerosal in vaccines was actually causing autism, then there should have been an obvious drop in autism rates following its removal, but there wasn’t (Madsen et al. 2003; Schechter and Grether 2008). Look at the autism rates over time for any of these countries, and you will not find a noticeable difference following the removal of thimerosal. That is extremely clear evidence that thimerosal does not cause autism.

Now, you may protest and say, “but aren’t trace amounts of it still present in some vaccines?” Yes, trace amounts are present in some vaccines, but think about what you just said, “trace amounts are present.” We are talking about less than 1 microgram of thimerosal per dose. Let me put that in perspective, a small paper clip weighs about 1,000,000 micrograms! We are talking about a dose that is much smaller than any study has ever found to be harmful, and, of course, the dose makes the poison. Finally, if thimerosal caused autism, then the shift to only trace amounts in a few vaccines should still have resulted in lower autism rates.

To quote Orac/Monty Python,

“The hypothesis that mercury in vaccines causes autism is about as dead a hypothesis as there can be. It’s passed on. The hypothesis is no more. It has ceased to be! It’s expired and gone to meet its maker. It’s a stiff. Bereft of life, it rests in peace. It’s shuffled off this mortal coil, run down the curtain and joined the bleedin’ choir invisible! This is an ex-hypothesis!”

Jump back to the top

Conference abstracts, case series opinions, other non-research
The lists contained several conference abstracts (which should always be considered preliminary and should not be used as evidence), case reports/series (which are glorified collections of anecdotes, and anecdotes aren’t evidence of causation), opinion papers, and other non-research papers (such as a bizarre, non-peer-reviewed student paper on court cases [Holland et al. 2011]).

I do, however, want to briefly talk about one of the case reports, because it is very famous in anti-vaccine circles. I am, of course, referring to Poling (2006). The paper documents the story of a girl who developed normally until she received a vaccine, then she regressed into autism. So it is every anti-vaccer’s story, just published in a scientific paper. However, the fact that it was published does not make a causal conclusion any more legitimate than if someone had said it on the internet. Saying, “X happened before Y, therefore X caused Y” is a logical fallacy known as post hoc ergo propter hoc. It does not prove or even suggest that X caused Y. There are also some other really disturbing aspects of this paper. For example, the subject of this study was Hannah Poling. That’s right, she is the author’s daughter (conflict of interest anyone?). In my opinion, parents publishing about their children’s health is probably not a great idea, especially when that parent proceeds to seek a financial settlement for their child’s health (as happened in this case).

Jump back to the top

Actual human studies on current vaccines and autism
After all of that filtering, we are down to just 14 studies and 8 non-systematic reviews that are actually on humans and are relevant to both autism and the current vaccine schedule.  So let’s look at them. Some of these are grouped together, and I have preceded each cluster with the title(s) of the paper(s) contained therein (titles are underlined and in quotes).

“A positive association found between autism prevalence and childhood vaccination uptake across the U.S. population”
“Empirical data confirm autism symptoms related to aluminum and acetaminophen exposure”
“Impact of environmental factors on the prevalence of autistic disorder after 1979”
“A comparison of temporal trends in United States autism prevalence to trends in suspected environmental factors”

organic food autism corrleation logical fallacy

Correlation does not equal causation. Organic food sales and autism rates are tightly correlated, but that does not mean that organic food causes autism. Image via the Genetic Literacy Project

First, I want to talk about DeLong (2011)Seneff et al. (2012), Deisher et al. (2014), and Nevison (2014). These four were association studies that looked for correlations between autism rates and vaccinations rates. That’s automatically a problem, because association does not mean causation, so even if these studies were good, they would not present evidence that vaccines cause autism. Also, there are additional problems. For example, the DeLong study used very crude state-wide data, failed to account for all possible confounders, and lumped speech disabilities in with autism for the analysis (more details on the problems with this paper here).

Similarly, Seneff et al. (2012) didn’t actual measure autism rates, but instead looked for mentions of it in the self-reported VAERS database (which went up over time), then tried to match that to the amount of aluminum in vaccines. Because it is self-reported, VAERS is notoriously unreliable for things like this, especially in this case because there is so much evidence that autism rates aren’t actually increasing (Rutter 2005; Taylor 2006; Bishop et al. 2008; Baxter et al. 2015; Hansen et al. 2015). Think about it, autism has become an increasingly hot topic over time, so you would naturally expect it to show up more often in VAERS over time, thus completely negating the study (other problems are described here).

Next, we have Deisher et al. (2014). This study attempted to look for correlations between the release of certain vaccines and increases in autism rates, and it is honestly a very hard paper to read because it is so horrible. The writing is nearly impossible to follow, the methods are nonsense, the statistics are total crap, the conclusions aren’t merited by those statistics, and the fundamental premises of the paper are refuted by numerous other studies (i.e., it assumes that autism rates are actually going up, which, once again, they likely aren’t). In short, what they did was take a data set that would normally be analyzed by linear regression, then chop it up based on cherry-picked data points. Other authors have explained the problems in detail, so I’ll defer to them (here and here).

Finally, we have Nevison (2014). In short, this study claimed to find an increase in autism rates over time (which is debatable), and it found that the increase correlates with glyphosate use and aluminum in vaccines. As illustrated earlier, however, you can also make a nice graph that shows a correlation between autism and organic food sales. Just because two things increase together does not mean that one causes the other. I think that you probably get the picture by now, so I won’t waste any more time on this paper.

“Do aluminum vaccine adjuvants contribute to the rising prevalence of autism? “

Now we arrive at the infamous Tomljenovic and Shaw (2011) study. It’s a paper that is so fundamentally flawed that I don’t even know where to begin, and it would take me an entire (and lengthy) post to go over all of the problems with it. Fortunately, others have done that for me, so I will defer you to them and just hit the highlights.

First, this study looked at two regressions: autism and aluminum in the US, and autism and aluminum across countries. We have lots of problems here. First, as Orac explains in more detail, you have a problem known as an ecological fallacy where you lump a very large data set (i.e., a country) into a single data point. In other words, they aren’t showing that individuals who receive vaccines have higher autism rates. Rather, they are showing that countries with lots of vaccines have higher autism rates. That is a huge problem because there are obviously tons of factors other than just vaccination rates that differ among countries (which means that we can’t be sure that vaccines are the thing that is causing the difference in autism rates). Additionally, as explained here, the sources of the data for different countries varied widely and involved different cohorts, and it appears that the authors cherry-picked their sources.

Even if we zoom in on the correlation in the US, we have serious problems. To assess autism rates, they looked at the number of autistic children who were reported via the Individuals with Disabilities Education Act (IDEA) database, which is an extremely problematic and inappropriate way to measure autism levels because it is affected by diagnostic changes, and, once again, the increase in autism seems to be largely due to diagnostic changes rather than actual changes (Rutter 2005; Taylor 2006; Bishop et al. 2008; Baxter et al. 2015; Hansen et al. 2015).

Finally, this is yet another association study. It cannot demonstrate causation, but that doesn’t stop the authors from trying. They misappropriate Hill’s criteria, which is a series of nine diagnostic criteria used to assess whether or not causation is likely. I will outline and discuss them below.

  1. Strength (robustness): Although they did get low P values, their methods have multiple problems and confounding factors that they did not account for. So this study fails the strength test.
  2. Consistency (consistent with other results): This study is extremely inconsistent with all of the large studies discussed early.
  3. Specificity (if you are dealing with a very specific, isolated event, causation is more likely): This was done at the country level. It is as far from specific as you can get.
  4. Temporality (cause happens before affect): This could not be assessed by this study.
  5. Biological gradient (i.e., higher dose = stronger effect): This was not evaluated by this study, but I cited several studies earlier that failed to find a relationship between the number of vaccine doses and autism.
  6. Plausibility: Once upon a time, the vaccine/autism hypothesis was plausible, but now that it has been so thoroughly tested, it is no longer plausible.
  7. Coherence (agreement of laboratory and epidemiological findings): There are lots of in vitro and animal trials that have found vaccines to be safe.
  8. Experiment: not relevant for this particular topic
  9. Analogy: not relevant for this particular topic

It’s pretty obvious that this paper epically fails Hill’s criteria. So, in short, this paper showed some shoddy correlations that were based on crude, inappropriate, and cherry-picked data sources. It absolutely is not evidence that vaccines cause autism.

“Hepatitis B triple series vaccine and developmental disability in US children aged 1-9 years”
“Hepatitis B vaccination of male neonates and autism diagnosis, NHIS 1997–2002.”

Next, I want to look at two studies by Gallagher and Goodman. Gallagher and Goodman (2008) is another study that used general developmental disorders, rather than autism specifically (which automatically makes it problematic and means that we can’t reach any conclusions specifically about autism). Further, it was yet another association study, and it relied on parental surveys (which are often prone to biases). Additionally, it had fairly small sample sizes (228 unvaccinated boys, 678 vaccinated boys, 217 unvaccinated girls, and 571 vaccinated girls). Most importantly though, its results are entertainingly problematic. It found higher levels of EIS (special education services) in vaccinated boys than unvaccinated boys (7% vs 3%), but it also found significantly lower levels of EIS in vaccinated girls than in unvaccinated girls (2% vs 6%). That is extremely clear evidence that the results of this study are simply statistical noise produced by a weak study design and small sample sizes. To put this another way, if you want to use this study as evidence that vaccines cause autism in boys, you must simultaneously use it as evidence that they prevent autism in girls.

Now, let’s look at Gallagher and Goodman 2010. This was a cross-sectional study, which is a study design that looks at the rate of something in a population, then looks for possible causes of that thing. This is a very weak type of association study which cannot establish causal relationships and is easily biased by numerous factors. Additionally, this study also had a very small sample size of only 31 boys with autism (which is what they used for the stats). The sample for non-autistic children was much higher, but the study is limited by the smallest sample size, and when you couple a weak experimental design with a tiny sample size, you get unreliable results, which is the best word to describe both Gallagher and Goodman studies: “unreliable.”

“Serological association of measles virus and human herpesvirus-6 with brain autoantibodies in Autism”
“Abnormal measles-mumps rubella antibodies and CNS autoimmunity in children with autism”

Next, let’s talk about two papers for which Vijendra Singh was the primary author. First, we have Singh et al. 1998, which is another really sorry excuse for a paper. It is yet another association study, and it also had tiny sample sizes (48 with autism and 43 controls; you should be detecting a theme by now). Most importantly, their numbers are so far off that I am willing to label it “fraudulent.” The authors looked at serum levels of HHV-6-IgG, measles-IgG, anti-MBP, and anti-NAFP, and found that 70% of the autistic children were positive for anti-MBP. Simple math tells us that 70% of 48 is 34 (after rounding). So 34 of their autistic children had anti-MBP; however, in table 1, where they are presenting the “associations” on which their entire paper is based, they reported that 37 autistic children had both measles IgG antibodies and anti-MBP. That is not possible if only 34 autistic children had anti-MBP. Similarly, they said that 57% of autistic children were positive for anti-NAFP. So 57% of 48 is 27 autistic children with anti-NAFP. Yet they claim that 30 autistic children had both measles IgG and anti-NAFP. Whenever you find inconsistencies of this magnitude in the core results of a paper, you should toss out the whole paper, because at that point, you don’t have any reason to trust anything in it. There are also lots of other problems with this study, such as the fact that there was no significant difference in viral levels in the autistic and non-autistic group (which is the opposite of what you would expect if exposure to the virus caused autism), but the numerical inconsistencies are so great that I don’t feel the need to go any further.

The next study from this group, (Singh et al. 2002) also looked at serum levels. Specifically, it was looking for antibodies produced by the MMR vaccine as well as anti-MBP. Once again, it is a small association study (125 autistic children, 92 controls), and the results are, unsurprisingly, a bit suspicious. They used two different techniques for detecting the antibodies, and the main one (which was used for the primary comparisons) found MMR antibodies in 60% of autistic children, or at least that’s what the text says. According to to figure 5, it was only about 55%. Again, inconsistencies like that in the main results are enormous red flags. At best, they mean that the authors were really sloppy (which should make you question every part of the study and analysis), and at worst, they are dishonest and fudged the results. Additionally, this technique did not detect MMR antibodies in any of the control children. This is extremely surprising, because all of the control children were vaccinated, which means that most of them should have had those antibodies. Further, their other technique (which was used on a subset of samples) did detect MMR antibodies in some of the controls, which means that either their primary method was not sensitive enough to be useful, or they lied. Either way, this paper is busted.

“Possible immunological disorders in autism: concomitant autoimmunity and immune tolerance”

Next, I want to talk about Kawashti. (2006). The inclusion of this paper in the anti-vaccers’ list amuses me to no end, because it is designed very similarly to the Singh et al. papers (i.e., it looked for antibodies from the MMR vaccine in children with and without autism), but it found the exact opposite! It found the antigens for measles, mumps, and rubella were present in 100% of non-autistic children, but only 50%, 73.3%, and 53.3% of autistic children. Further, it concluded “At this stage, we can conclude that M.M.R. vaccine may not be a cause of autism.” That’s right, this is not actually an anti-vaccine paper. This, once again, shows just how little quality control went into constructing the anti-vaccine lists. This paper directly contradicts other papers in the list, yet it was still included (note: this is another small association study, so I’m not touting it as evidence that vaccines don’t cause autism; rather, the point is simply that the anti-vaccers screwed up and put a pro-vaccine paper in their list).

“Preterm birth, vaccination and neurodevelopmental disorders: a cross-sectional study of 6- to 12-year-old vaccinated and unvaccinated children”
“Pilot comparative study on the health of vaccinated and unvaccinated 6- to 12- year old U.S. children”

Next, let’s talk about a particularly awful pair of studies: Mawson et al. (2017a, b). These papers aren’t actually published in real, respected journals. Rather, they both appear in the predatory journal, “Journal of Translational Science.” Further, one of them (Mawson et al. 2017a) only appeared there after being retracted from a different journal. Retraction Watch reported that Translational Science was going to retract it as well, but it is currently still available on their site.

Both “studies” are based on an online survey that used a method known as “convenience sampling.” They simply sent the survey to homeschool groups (which tend to be dominated by anti-vaccers), and they did not use any of the controls or randomizations that are neccissary for a proper survey. In statistics courses, literally the first thing that you learn about surveys is that you shouldn’t use convience sampling.

Essentially, what these studies did was poll anti-vaccers about their children’s medical history. Unsurprisingly, when you poll people who are convinced that vaccines cause autsim, you get the result that vaccines cause autism. These papers are complete nonsense (which is why they are in a predatory journal instead of real journals). You can read more about these studies at Science-Based Medicne, Vaccines Work, and Respectful Insolance.

“Can awareness of medical pathophysiology in autism lead to primary care autism prevention strategies?”

Now, I want to shift gears a bit and talk about Mumper (2013). This is a truly bizarre “study.” It was published in the North American Journal of Medicine and Science, which is a journal that is so minor that I couldn’t even find an impact factor (in laymen’s terms, that means that no one cites this journal or takes it seriously). Further, the paper itself doesn’t follow any of the standard conventions for a scientific paper (it includes bizarre sections like anecdotes of the author’s travels to other countries). It looks more like an undergraduate report than a scientific paper.

It may seem like I am being nitpicky, but all of those things are actually clues that this is a subpar paper that did not go through a proper review process. You should learn to watch out for flags like that as you read scientific papers. Then, of course, we have the actual experimental design (and I’m using that term very loosely). The author works at a private pediatric clinic that stresses a lot of different things like breastfeeding, probiotics, nutritional counseling, flexible vaccine schedules, etc. Some of those are good, some are bad, but what she did, was go back through her records since 2005 to see if any of their infants had been diagnosed with autism while under their care. She found that out of 294 infants, none had been diagnosed with autism. This is lower than the national background rate of 1 in 59, so she made an astronomical jump to the conclusion that her clinic’s practices prevent autism (or at least avoid the causes of autism). That’s clearly an absurd conclusion. What she did is not even close to a proper study. Nevertheless, let’s assume for a second that it was. Let’s assume that there is actually something happening at that clinic that prevents autism. You still can’t jump to the conclusion that it was vaccines because there are so many factors. Maybe it was breastfeeding. Maybe it was probiotics, etc. This is not a real study. It is an anecdote that has been dressed up as a study.

“Detection and sequencing of measles virus from peripheral mononuclear cells from patients with inflammatory bowel disease and autism.”

Kawashima et al. (2000) simply found DNA from the measles vaccine in the colons of nine autistic patients, which of course doesn’t mean anything other than that DNA from the measles vaccine was in their colons. That result is not even remotely evidence that the vaccine does anything harmful.

“Epidemiologic and molecular relationship between vaccine manufacture and autism spectrum disorder prevalence.”

This paper (Deisher et al. 2015) was thoroughly debunked on Science-Based Medicine, so I will defer to that post, but in short, it is on an implausible hypothesis, the data were collected in a very questionable way, and the data were presented deceptively to show trends that aren’t really there. Additionally, this paper was on the MMR vaccine, which is also the vaccine that was specifically examined by most of the large cohort studies, the massive meta-analysis, etc. All of those studies were far more powerful than this study, so you really can’t use this paper as evidence that those papers are wrong. Finally, this paper was published in an extremely minor journal (the impact factor is roughly 0), which is a huge red flag that it is junk science.

Note: This paper was not in any of the anti-vaccine lists so it is not included in the paper count, but I have seen anti-vaccers citing it elsewhere, so I have included it here

Jump back to the top

Reviews
At this point, I have gone through the relevant anti-vaccine literature and showed that all of the studies are small and incapable of demonstrating causation, and I have showed that many of them are riddled with problems. So you may be wondering, “what could the reviews possibly be on?” Well this is where we come back to the importance of systematic reviews. If you recall in the pro-vaccine section, all of the reviews that I cited were systematic, meaning that they used pre-defined search terms and criteria. The anti-vaccine reviews, however, are not systematic. Thus, they completely ignored all of the large studies that have discredited the vaccine/autism hypothesis, they ignored all of the problems with the anti-vaccine studies, and they used all of those tiny, problematic studies to spin a fanciful tale in which, through a complex, unlikely, and thoroughly discredited series of events, vaccines cause autism. In other words, all that these “reviews” do is set forth a hypothesis. Doing that would be fine if it wasn’t for the fact that the hypothesis has been discredited by numerous massive studies. Again, you always reject a hypothesis based on the evidence. You never reject the evidence based on a hypothesis, but that is exactly what these papers are doing. They are deceptively only showing the papers that support their position while ignoring all of the papers that refute it.

Here is the list of reviews: Rimland and McGinnis. 2002, Singh. 2009, Ratajczak 2011 (details of problems here and here), Sienkiewicx et al. 2012 (details of problems here) Shaw and Tomljenovic. 2013b, Shaw et al. 2014a, Morris et al. 2017a and Lyons-Weiler 2018 (details of problems here). Most of these contain numerous blatantly false (or at least highly misleading) statements, but the main point that I want to drive home is simply that they are stating hypotheses rather than presenting evidence of causation, and they are ignoring the fact that those hypotheses have already been tested and refuted.

Note: to be fair, Rimland and McGinnis (2002) was written before the large studies were available, so they were stating a hypothesis that had yet to be thoroughly tested, which is fine, but most of these reviews were written after the hypothesis had been thoroughly refuted.

Note: There were actually 37 reviews in all, but most of them were not about vaccines, were about thimerosal, etc. So only the eight listed here were actually relevant for the current vaccine schedule.

Jump back to the top

Conflicts of interest
Anti-vaccers are very quick to point out conflicts of interest in pro-vaccine papers (or, more often, assume that they exist before they even check for them), but they are very slow to acknowledge them in their own papers. So I want to briefly provide a few examples to show that they do in fact exist (I did not check for conflicts in all of the anti-vaccine papers).

First, it is worth mentioning that this whole vaccine/autism mess got started when Andrew Wakefield falsified evidence against the MMR vaccine and conspired with lawyers to sue pharmaceutical companies, all while covertly working on a patent for his own vaccine which he planned to replace the MMR vaccine with (details here and here). Similarly, Geier, who is the author of a case series that anti-vaccers love to cite (Geier and Geier 2007) as well as several other anti-vaccine studies, has had his medical license suspended for unethical behavior and, “incompetence or multiple instances of negligence.” The heroes of the anti-vaccine movement leave much to be desired. Further, those two are far from alone. As I mentioned earlier, Dr. Poling published his infamous case report while in the process of seeking a financial settlement for his daughter’s “vaccine injury.”

Dr. Shaw and Dr. Tomljenovic are also two fantastic examples of conflicts of interest. These are two of the most prominent anti-vaccine scientists, and eight of the papers in the anti-vaccers’ lists were authored by at least one of them. However, both of them have served as consultants or expert witnesses in vaccine lawsuits, Shaw is the chair of the Scientific Advisory Board for an anti-vaccine group, and at least one of their studies was funded by members of the governing board of that group (more info here). Funding from an activist group that describes vaccines as, “a holocaust of poison on our children’s brains and immune systems” is just as big of a conflict of interest as funding from a pharmaceutical company. So, according to standard anti-vax reasoning, this should cast doubt on all of the authors’ work.

Similarly, Dr. Singh (who you may remember authored two of the papers and one of the reviews that I talked about) was funded by the Autism Research Institute, which, at the time that he received funding, ran a program called “Defeat Autism Now!,” which actively promoted the idea that vaccines cause autism. Further, remember that horrible DeLong (2011) paper that I talked about earlier? Well DeLong is a board member of the prominent anti-vaccine group “SafeMinds,” and, like Poling, is the parent of an autistic child. Similarly, those Mawson surveys I talked about were funded by Generation Rescue, yet another anti-vaccine organization.

I could keep going, but I think that I have made my point clear. Anti-vaccers like to pretend that all of their studies are conflict free and represent true, unbiased research. In reality, there are plenty of anti-vaccine organizations that are happy to fund anti-vaccine studies, and many of the authors are deeply involved in the anti-vaccine movement. To be clear, I’m not suggesting that we should automatically reject these papers because of those conflicts (we should reject them because they are junk science), but I wanted to point out that, despite what anti-vaccers like to think, they are not free from conflicts of interest.

Note: please see these two posts for more info on when and how it is OK to attack a source.

Jump back to the top

Studies that I did not include
You may have noticed that I did not talk about either the original Wakefield study or Hooker (2014). That is because both of these studies were so flawed that they were retracted (I talked more about Hooker’s study here).

All of the anti-vaccine papers fall into the lowest categories of evidence, and none of them were capable of showing causal relationships (details here).

All of the anti-vaccine papers fell into the lowest categories of evidence, and none of them were capable of showing causal relationships (details here).

Overview of anti-vaccine papers
In short, the vast majority of papers cited by anti-vaccers aren’t even about vaccines and autism. Of the ones that are, many of them are animal trials and in vitro trials, or they are about thimerosal, which is no longer in vaccines and has been documented to be safe via several very large epidemiological studies. Indeed, there were only 14 studies on humans that were about both vaccines and autism and were relevant to the current vaccine schedule, but none of them were case-controlled or cohort studies, and there were no meta-analyses or systematic reviews. The studies used small sample sizes and shoddy statistics to show crude correlations, and none of them had the ability to assign causation. Further, most of them were filled with problems, and one of them was actually a pro-vaccine paper. There were also eight non-systematic reviews, but these were essentially glorified opinion papers that ignored all of the literature against a link between autism and vaccines. They simply presented hypotheses rather than evidence, and those hypotheses have been thoroughly discredited by large studies.

Jump back to the top

Anecdotes, Court Cases, VAERS, etc.
At this point, it should be exceptionally clear that there is no good scientific evidence to suggest that vaccines cause autism, which means that this is usually the point at which people start trying to use non-scientific evidence. Therefore, I want to very briefly explain the problems with some common non-scientific arguments.

The most common response to the statement, “vaccines don’t cause autism” is probably the appeal to anecdotes. The internet is full of stories of people who vaccinated their child, then watched as the child regressed. The problem is, of course, that anecdotes aren’t evidence. The fact that X happened before Y does not mean that X caused Y. In fact, arguing that X caused Y is a logical fallacy known as post hoc ergo propter hoc. This is especially true for something like autism because the first signs of autism usually appear around the same age as many vaccinations. Therefore, given the large number of people who develop autism, and the large number of people who are vaccinated, you expect there to be many cases where vaccination and the onset of autism occur together just by chance. Really think about that for a minute. If 1 in 59 children will have autism, usually with the first obvious symptoms occurring around 2-3 years of age, and over 90% of children receive vaccines around 2-3 years of age, then there should be lots of cases where parents notice the signs of autism shortly after vaccinating, even if vaccines aren’t the cause (I actually ran the numbers on this here).

I fully understand why parents would blame vaccines. I understand why seeing your child develop autism shortly after receiving a vaccine would make you think that the vaccine was responsible, but you need to realize that lots of things happen together just by chance, and, as a result, anecdotes are not valid evidence. If vaccines actually did cause autism at the rates claimed by anti-vaccers, then the large, systematic studies should have found a significant difference between vaccinated and unvaccinated children. In the face of evidence like that, it is not logically or scientifically valid to cling to anecdotes.

An extension of the anecdote argument is to cite reports of vaccines causing autism in the Vaccine Adverse Event Reporting System (VAERS). VAERS is, however, a self reported database. In other words, it is just a collection of anecdotes. The fact that someone reported an anecdote to VAERS doesn’t make the anecdote any more trustworthy than if it was on Facebook. The point of databases like this is to allow doctors and scientists to identify potential issues that need to be studied. It is not meant to be used as evidence of causation (more details about issues with using VAERS as evidence here and here).

A final line of anecdotal reasoning involves appealing to the vaccine package inserts (which is the one and only time that anti-vaccers trust pharmaceutical companies). The problem is, once again, that the lists of adverse reactions do not demonstrate causation. Those lists consist of any symptoms reported during clinical trials, most of which were almost certainly not caused by the vaccine. In fact, the package inserts even state that the list of adverse reactions is not a list of confirmed causal relationships.

Another common strategy is to appeal to court cases. There have been various court cases in various countries where money has been awarded to people who claim that vaccines gave their child autism (there have also been plenty that were thrown out of court or later overturned). To anti-vaccers, these are admissions of guilt by governments and confirm that vaccines are dangerous. In reality, they are nothing of the kind. The fact that a judge concluded that a vaccine caused autism does not mean that the vaccine actually did cause autism. This is a blatant appeal to authority fallacy. Judges aren’t infallible, and they usually aren’t even scientists. A judge can be deceived about the current state of our scientific knowledge just as easily as anyone else. So you absolutely cannot say, “This court gave money for an autism case, therefore all of those massive studies with hundreds of thousands of children must be wrong.” Hopefully you can see why statements like that are absurd (more on why court cases shouldn’t be used as scientific evidence here).

Additionally, there has been a great deal of fuss over the “CDC whistleblower.”  I won’t go into the details because it is just too long of a story, but the short version is that there was no cover up and the CDC did not hide evidence or deceive the public. This argument is nothing more than typical conspiracy theorist ramblings. Many others have explained the situation in detail, so if you are prone to using this argument, please see their posts (for example, here, here, and here).

Finally, when faced with the overwhelming evidence that vaccines are safe, many people try to argue that science shouldn’t be trusted because it has been wrong before (debunked here) or by citing the work of Dr. Ioannidis and claiming that most studies are wrong, therefore we can blindly ignore the studies on vaccine safety (debunked here). Both of these arguments are logically and scientifically invalid. The fact that science has been wrong before doesn’t mean that you can dismiss it anytime that you want to. Similarly, what Ioannidis’s work actually shows is exactly what I have argued in this post: on any topic there will be many small, poorly conducted, and unreliable studies, so we have to really on a consensus of large, high-quality studies.

Conclusion

If you made it all the way here, then congratulations on reading the world’s longest blog post (or at least an unusually long post). It should now be clear to you that the evidence really is overwhelmingly supportive of vaccines. Even though anti-vaccers claim to have lengthy lists of papers supporting their position, most of those papers are irrelevant, used weak designs, and had small sample sizes. In contrast, the literature supporting vaccine safety consists of multiple exceptionally large and powerful studies. So there really is no good scientific evidence to suggest that vaccines cause autism.

Jump back to the top

Literature Cited

  • Akinrinade et al. 2015. Interplay of glia activation and oxidative stress formation in fluoride and aluminium exposure. Pathophysiology 22:39–48.
  • Anders et al. 2004. Thimerosal exposure in infants and developmental disorders: a retrospective cohort study in the United Kingdom does not support a causal association. Pediatrics 114:584–591
  • Austin and Shandley. 2008. An investigation of porphyrinuria in Australian children with autism. J Toxicol Environ Health A 71:1349–1351
  • Baxter et al. 2015. The epidemiology and global burden of autism spectrum disorders. Psychological Medicine 45:601–613
  • Bernard et al. 2001. Autism: a novel form of mercury poisoning. Medical Hypotheses 56:462–471.
  • Bishop et al. 2008. Autism and diagnostic substitution: evidence from a study of adults with a history of developmental language disorder. Dev Med Child Neurol 50: 341–345
  • Blaurock-Busch et al. 2012. Toxic metals and essential elements in hair and severity of symptoms among children with autism. Maedica 7:38–48
  • Blaxill. 2004. What’s going on? The question of time trends in autism. Public Health Rep. 119:536–551
  • Blaxill et al. 2003. Commentary on Croen et al. (2002), The Changing Prevalence of Autism in California. J Autism Dev Disord. 223–226
  • Blaxill et al. 2004. Thimerosal and autism? A plausible hypothesis that should not be dismissed. Med Hypotheses 62:788–794
  • Branch. 2009. Gender-selective toxicity of thimerosal. Exp Toxicol Pathol 61:133–136
  • Breecea et al. 2013. Myeloid dendritic cells frequencies are increased in children with autism spectrum disorder and associated with amygdala volume and repetitive behaviors. Brain Behav Immunity 31:69–75
  • Brown et al. 2014. Elevated maternal C-reactive protein and autism in a national birth cohort. Mol Psychiatry 19:259–264
  • Burbacher. 2005. Comparison of blood and brain mercury levels in infant monkeys exposed to methylmercury or vaccines containing thimerosal. Environ Health Perspect 113:1015–1021
  • Carvalho et al. 2008. Inhibition of the human thioredoxin system. A molecular mechanism of mercury toxicity. J Biol Chem 283:11913-11923
  • Carvalho et al. 2011. Effects of selenite and chelating agents on mammalian thioredoxin reductase inhibited by mercury: implications for treatment of mercury poisoning. FASEB J 25:370–381
  • Charleston et al. 1994. Increases in the number of reactive glia in the visual cortex of Macaca fascicularis following subclinical long-term methyl mercury exposure. Toxicol Appl Pharmacol 129:196–206
  • Chauhan and Chauhan. 2006. Oxidative stress in autism. Pathophysiology 13:171–181
  • Chen et al. 2004. No evidence for links between autism, MMR and measles virus. Psychol Med 34:543–553
  • Chen et al. 2013. Effect of thimerosal on the neurodevelopment of premature rats. World J. Pediatr. 9:356–360.
  • Cheuk and Wong. 2006. Attention-deficit hyperactivity disorder and blood mercury level: a case-control study in Chinese children. Neuropediatris 37:234–240
  • Christian et al. 2011. Inflammatory responses to trivalent influenza virus vaccine among pregnant women. Vaccine 29:8982–8987
  • Dales et al. 2001. Time trends in autism and in MMR immunization coverage in California. JAMA 285:1183–1185
  • Deisher et al. 2014.  Impact of environmental factors on the prevalence of autistic disorder after 1979. J Public Health Epidemiol 6:271–286
  • Deisher et al. 2015. Epidemiologic and molecular relationship between vaccine manufacture and autism spectrum disorder prevalence. Issues in Law and Medicine 30:47-70.
  • DeLong. 2011. A positive association found between autism prevalence and childhood vaccination uptake across the U.S. population. J Toxicol Environ Health A 74:903-916
  • Demicheli et al. 2012. Vaccines for measles, mumps and rubella in children. Cochrane Database Syst Rev
  • DeStefano et al. 2004. Age at first measles-mumps-rubella vaccination in children with autism and school-matched control subjects: a population-based study in metropolitan Atlanta. Pediatrics 113:259–266
  • DeStefano et al. 2013. Increasing exposure to antibody-stimulating proteins and polysaccharides in vaccines is not associated with risk of autism. J Ped 163:561–567
  • Desoto and Hitlan. 2007. Blood levels of mercury are related to diagnosis of autism: a reanalysis of an important data set. J Child Neurol 22:1308-1311
  • Desoto and Hitlan. 2010. Sorting out the spinning of autism: heavy metals and the question of incidence. Acta Neurobiol Exp 70:165–176
  • Dickerson et al. 2015. Autism spectrum disorder prevalence and proximity to industrial facilities releasing arsenic, lead or mercury. Sci Total Environ 536:245–251
  • Dórea. 2011. Integrating experimental (in vitro and in vivo) neurotoxicity studies of low-dose thimerosal relevant to vaccines. Neurochem Res 36:927–938
  • Duszczyk-Budhathoki et al. 2012. Administration of thimerosal to infant rats increases overflow of glutamate and aspartate in the prefrontal cortex: protective role of dehydroepiandrosterone sulfate. Neurochem 37:436–447
  • Ekstrand et al. 2010. Mercury toxixokenetics—dependency on strain and gender. Toxicol Appl Pharmacol 15:283–291
  • Elsabbagh et al. 2012. Global prevalence of autism and other pervasive developmental disorders. Autism Research 5:160–179
  • Ercal et al. 2001. Toxic metals and oxidative stress part 1: mechanisms involved in metal induced oxidative stress. Curr Top Med Chem 1:529–539
  • Eshraghi et al. 2018. Early Disruption of the Microbiome Leading to Decreased Antioxidant Capacity and Epigenetic Changes: Implications for the Rise in Autism. Front Cell Neurosci
  • Ewing. 2009. What is regressive autism and why does it occur? Is it the consequence of multi-systemic dysfunction affecting the elimination of heavy metals and the ability to regulate neural temperature? N AM J Med Sci 1:28–47
  • Feenstra et al. 2014. Common variants associated with general and MMR vaccine-related febrile seizures. Nat. Genet. 46:1274–1282
  • Fiorentino et al. 2016. Blood–brain barrier and intestinal epithelial barrier alterations in autism spectrum disorders. Molecular Autism 7:49
  • Fombonne et al. 2006. Pervasive Developmental Disorders in Montreal, Quebec, Canada: Prevalence and Links With Immunizations. Pediatrics 118
  • Gagag et al. 2015. Administration of thimerosal-containing vaccines to infant rhesus macaques does not result in autism-like behavior or neuropathology. PNAS 112: 12498–12503
  • Gallagher and Goodman. 2008. Hepatitis B triple series vaccine and developmental disability in US children aged 1-9 years. Toxicol Environ Chem 90:997–1008
    Gallagher and Goodman. 2010. Hepatitis B vaccination of male neonates and autism diagnosis, NHIS 1997–2002. J Toxicol Environ Health A 73:1665–1677
  • Gargus and Imtiaz. 2008. Mitochondrial energy-deficient endophenotype in autism. Am J Biochem Biotech 4:198–207
  • Gesundheit et al. 2013. Immunological and autoimmune considerations of Autism Spectrum Disorders. J. Autoimmun 44:1–7
  • Geier and Geier. 2007. A case series of children with apparent mercury toxic encephalopathies manifesting with clinical symptoms of regressive autistic disorder. J Toxicol Environ Health A 15:837–851
  • Ghanizadeh et al. 2012. Glutathione-related factors and oxidative stress in autism, a review. Curr Med Chem 19:4000–4005
  • Goin-Kochel et al. 2016. Parental report of vaccine receipt in children with autism spectrum disorder: Do rates differ by pattern of ASD onset? Vaccine 34:1335­–1342.
  • Gorrindo et al. 2013. Enrichment of elevated plasma F2t-Isoprostane levels in individuals with autism who are stratified by presence of gastrointestinal dysfunction. PLoS ONE 8:e68444
  • Goth 2006. Uncoupling of ATP-mediated calcium signalling and dysregulated interleukin-6 secretion in dendritic cells by nanomolar thimerosal. Environ Health Perspect 114:1083–1091
  • Griffiths and Levy. 2017. Evidence of Mitochondrial Dysfunction in Autism: Biochemical Links, Genetic-Based Associations, and Non-Energy-Related Mechanisms. Oxid Med Cell Longev 2017:4314025
  • Gupta. 2014. Transcriptome analysis reveals dysregulation of innate immune response genes and neuronal activity-dependent genes in autism. Nat Commun 5:5748
  • Guy et al. 2015. Infants born late/moderately preterm are at increased risk for a positive autism screen at 2 years of age. J. Pediatr 166:269–275
  • Haley 2005. Mercury toxicity: Genetic susceptibility and synergistic effects. Medical Veritas 2:535-542
  • Hamza et al. 2012. Hepatitis B vaccine induces apoptotic death in Hepa1-6 cells. Apoptosis 17:516–527
  • Hansen et al. 2015. Explaining the increase in the prevalence of autism spectrum disorders: the proportion attributable to changes in reporting practices. JAMA Pediatrics 169:56–62
  • Herbert. 2005a. Autism: a brain disorder, or a disorder that affects the brain? Clinical Neurophsychiatry 2:354–379
  • Herbert. 2005b. Large brains in autism: the challenge of pervasive abnormality. Neuroscientist. 11:417–40
  • Hertz-Picciotto and Delwiche. 2009. The rise in autism and the role of age at diagnosis. Epidemiology 20:84–90
  • Herdman et al. 2006. Thimerosal induces apoptosis in a neuroblastoma model via the cJun N-terminal kinase pathway. Toxicol Sci 92:246–253
  • Hewitson et al. 2008. Pediatric vaccines influence primate behaviour and amygdala growth and opioid ligand binding. International Meeting for Autism Research
  • Hewitson et al. 2010. Influence of pediatric vaccines on amygdala growth and opioid ligand binding in rhesus macaque infants: A pilot study. Acta Neurobiol Exp 70:147–164
  • Hviid et al. 2019. Measles, mumps, rubella vaccination and autism: A nationwide cohort study. Annals of Internal Medicine.
  • Holland et al. 2011. Unanswered questions: A review of compensated cases of vaccine-induced brain injury. Pace Environmental Law Review 28:2
  • Holmes. 2003. Reduced levels of mercury in first baby haircuts of autistic children. Int J Toxicol 22:277–285
  • Honda et al. 2005. No effect of MMR withdrawal on the incidence of autism: a total population study. J Child Psych and Psychiat 46:572–579
  • RETRACTED Hooker. 2014. Measles-mumps-rubella vaccination timing and autism among young African American boys: a reanalysis of CDC data. Translational Neurodegeneration 3:16
  • Howsmon et al. 2017. Classification and adaptive behavior prediction of children with autism spectrum disorder based upon multivariate data analysis of markers of oxidative stress and DNA methylation. PLoS Computational Biology
  • Howsmon et al. 2018. Multivariate techniques enable a biochemical classification of children with autism spectrum disorder versus typically‐developing peers: A comparison and validation study. Bioengineering and Translational Medicine 3:156–165.
  • Humphrey et al. 2005. Mitochondrial mediated thimerosal-induced apoptosis in a human neuroblastoma cell line (SK-N-SH). Neurotoxicology. 26:407–416
  • Hurley et al. 2010. Thimerosal-containing vaccines and autism: a review of recent epidemiologic studies. J. Pediatr Pharmacol Ther 15:173–181
  • Hviid et al. 2003. Association between thimerosal-containing vaccine and autism. JAMA 290:1763–1766
  • WITHDRAWN Inbar, et al. 2016. Behavioral abnormalities in young female mice following administration of aluminium adjuvants and the human papillomavirus (HPV) vaccine Gardasil. Vaccine
  • Jain et al. 2015. Autism occurrence by MMR vaccine status among US children with older siblings with and without autism. JAMA 313:1534–1540
  • Jafari et al. 2017. The association between mercury levels and autism spectrum disorders: A systematic review and meta-analysis. Journal of Trace Elements in Medicine and Biology 44:289–297.
  • James 2004. Metabolic biomarkers of increased oxidative stress and impaired methylation capacity in children with autism. Am J Clin Nutr 80:1611–1617
  • James et al. 2005. Thimerosal neurotoxicity is associated with glutathione depletion: protection with glutathione precursors. NeuroToxicol 26:1–8
  • James et al. 2006. Metabolic endophenotype and related genotypes are associated with oxidative stress in children with autism. American Journal of Medical Genetics B 141B:947–956.
  • Jefferson et al. 2003. Unintended events following immunization with MMR: a systematic review. Vaccine 21:3954–3960
  • Kanner. 1943. Autistic disturbances of affective contact. Nervous Child 2:217–250
  • Kawashima et al. 2000. Detection and sequencing of measles virus from peripheral mononuclear cells from patients with inflammatory bowel disease and autism. Dig Dis Sci 45:723–729
  • Kawashti. 2006. Possible immunological disorders in autism: concomitant autoimmunity and immune tolerance. Egypt J Immunol 13:99–104
  • Kaye et al. 2001. Mumps, measles, and rubella vaccine and the incidence of autism recorded by general practitioners: a time-trend analysis. West J Med 174387–390
  • Kempuraj. 2010. Mercury induces inflammatory mediator release from human mast cells. J Neuroinflammation 7:20
  • Kern and Jones. 2006. Evidence of toxicity, oxidative stress, and neuronal insult in autism. J Toxicol Environ Health B Crit Rev 9:485–499
  • Khan et al. 2013. Slow CCL2-dependent translocation of biopersistent particles from muscle to brain. BMC Med 11:99
  • Khaled et al. 2016. Altered urinary porphyrins and mercury exposure as biomarkers for autism severity in Egyptian children with autism spectrum disorder. Metabolic Brain Disease 31:1419–1426.
  • King et al. 2013. Topoisomerases facilitate transcription of long genes linked to autism. Nature 501:58–62
  • Klein and Diehl. 2004. Relationship between MMR vaccine and autism. Annals of Pharmacotherapy 38:1297–1300
  • Leviton et al. 2018. The risk of neurodevelopmental disorders at age 10 years associated with blood concentrations of interleukins 4 and 10 during the first postnatal month of children born extremely preterm. Cytokine 110:181–188.
  • Li et al. 2014. Transcriptomic analyses of neurotoxic effects in mouse brain after intermittent neonatal administration of thimerosal. Toxicol Sci 139:452–465
  • Li et al. 2018. Blood Mercury, Arsenic, Cadmium, and Lead in Children with Autism Spectrum Disorder. Biological Trace Element Research 181:31–37.
  • Lukiw et al. 2005. Nanomolar aluminum induces pro-inflammatory and pro-apoptotic gene expression in human brain cells in primary culture. J Inorg Biochem 99:1895–1898
  • Lyons-Weiler 2018. Autism is an Acquired Cellular Detoxification Deficiency Syndrome with Heterogeneous Genetic Predisposition. Autism Open Access 8:1
  • Madsen et al. 2002. A population-based study of measles, mumps, and rubella vaccination and autism. New England Journal of Medicine 347:1477–1482
  • Madsen et al. 2003. Thimerosal and the occurrence of autism: negative ecological evidence from Danish population-based data. Pediatrics 112:604–606
  • Makela et al. 2002. Neurologic disorders after measles-mumps-rubella vaccination. Pediatrics 110:957­–963.
  • Margaret et al. 2014. Safety of vaccines used for routine immunization of US children: a systematic review. Pediatrics 134:1–13
  • RETRACTED Mawson et al. 2017a. Pilot comparative study on the health of vaccinated and unvaccinated 6- to 12- year old U.S. children. Journal of Translational Science 3. (see retraction watch)
  • Mawson et al. 2017b. Preterm birth, vaccination and neurodevelopmental disorders: a cross-sectional study of 6- to 12-year-old vaccinated and unvaccinated children. Journal of Translational Science 3.
  • Migdal et al. 2010. Sensitization effect of thimerosal is mediated in vitro via reactive oxygen species and calcium signalling. Toxicol. 274:1–9
  • Minami et al. 2010. Induction of metallothionein in mouse cerebellum and cerebrum with low-dose thimerosal injection. Cell Biol Toxicol 26:143–152
  • Mizejewski et al. 2013. Newborn screening for autism: in search of candidate biomarkers. Biomarkers in medicine 7:247–260.
  • Molina and Shoenfeld 2005. Infection, vaccines and other environmental triggers of autoimmunity. Autoimmunity 38:235–245
  • Mold et al. 2018. Aluminium in brain tissue in autism. Journal of Trace Elements in Medicine and Biology 46:76–82.
  • Morgan et al. 2010. Microglial Activation and Increased Microglial Density Observed in the Dorsolateral Prefrontal Cortex in Autism. Biological Psychiatry 68:386–376.
  • Morris et al. 2017a. The putative role of environmental aluminium in the development of chronic neuropathology in adults and children. How strong is the evidence and what could be the mechanisms involved? Metabolic Brain Disease 32:1335–1355.
  • Morris et al. 2017b. The Putative Role of Environmental Mercury in the Pathogenesis and Pathophysiology of Autism Spectrum Disorders and Subtypes. Molecular Neurobiology 55: 4834­–4856
  • Mumper. 2013. Can awareness of medical pathophysiology in autism lead to primary care autism prevention strategies? N A J Med Sci 6:134–144
  • Naik et al. 2011. A Study of Nuclear Transcription Factor-Kappa B in Childhood Autism. 6:e19488
  • Namvarpour et al. 2018. Protective role of alpha-lipoic acid in impairments of social and stereotyped behaviors induced by early postnatal administration of thimerosal in male rat. Neurotoxicology Teratology 67:1–9.
  • Nataf. 2006. Porphyrinuria in childhood autistic disorder: implications for environmental toxicity. Toxicol Appl Pharmacol 2006:99–108
  • Neustadt and Pieczenik. 2007. Heavy-metal toxicity – with emphasis on mercury. IMCJ 6:26–32
  • Nevison 2014. A comparison of temporal trends in United States autism prevalence to trends in suspected environmental factors. Envrion Health 13:73
  • Nevison et al. 2017. Diagnostic Substitution for Intellectual Disability: A Flawed Explanation for the Rise in Autism. Journal of Autism and Developmental Disorders 47:2733–2742.
  • Nevison et al. 2018. California Autism Prevalence Trends from 1931 to 2014 and Comparison to National ASD Data from IDEA and ADDM. Journal of Autism and Developmental Disorders 48:4103–4117.
  • Nordahl et al. 2011. Brain enlargement is associated with regression in preschool-age boys with autism spectrum disorders. Proceedings of the National Academy of Sciences 108:20195–20200.
  • Olczak et al. 2009. Neonatal administration of a vaccine preservative, thimerosal, produces lasting impairment of nociception and apparent activation of opioid system in rats. Brain Research 1301:143–151
  • Olczak et al. 2010a. Lasting neuropathological changes in rat brain after intermittent neonatal administration of thimerosal. Folia Neuropathologica 48:258–269.
  • Olczak et al. 2010b. Neonatal administration of thimerosal causes persistent changes in mu opioid receptors in the rat brain. Neurochem Res 35:1840–1847
  • Olczak et al. 2011. Persistent behavioral impairments and alterations of brain dopamine system after early postnatal administration of thimerosal in rats. Behavioral Brain Research 30:107–118.
  • Oliveira. 2007. Epidemiology of autism spectrum disorder in Portugal: prevalence, clinical characterization, and medical conditions. Dev Med Child Neurol 49:726–733
  • Palmer et al. 2006. Environmental mercury release, special education rates, and autism disorder: an ecological study of Texas. Health Place 12:203–209
  • Palmer et al. 2008. Proximity to point sources of environmental mercury release as a predictor of autism prevalence. Health Place 15:18–24
  • Parker et al. 2004. Thimerosal-containing vaccines and autistic spectrum disorder: A critical review of published original data. Pediatrics 114:793–804
  • Petrik. 2007. Aluminum adjuvant linked to Gulf War illness induces motor neuron death in mice. Neuromolecular Med 9:83–100
  • Poling. 2006. Developmental regression and mitochondrial dysfunction in a child with autism. J Child Neurol 21:170–172
  • Price et al. 2010. Prenatal and infant exposure to thimerosal from vaccines and immunoglobulins and risk of autism. Pediatrics 16:656–64
  • Ratajczak. 2011. Theoretical aspects of autism: causes–a review. J Immunotoxicol 8:68–79
  • Rice. 1989. Brain and tissue levels of mercury after chronic methylmercury exposure in the monkey. J Toxicol Environ Health 27:189–198
  • Rimland and McGinnis. 2002. Vaccines and autism. Laboratory Medicine 9:708–717
  • Risher and Tucker 2017. Alkyl Mercury-Induced Toxicity: Multiple Mechanisms of Action. Reviews of Environmental Contamination and Toxicology 240:105–149.
  • Robertson et al. 2015. Reduced GABAergic action in the autistic brain. Curr Biol 26:80–85
  • Rooney. 2014. The retention time of inorganic mercury in the brain—a systematic review of the evidence. Toxicol Appl Pharmacol 274:425–435
  • Rossignol and Bradstreet. 2008. Evidence of mitochondrial dysfunction in autism and implications for treatment. Am J Biochem Biotechn 4:208–217
  • Rossignol and Frye. 2012. Mitochondrial dysfunction in autism spectrum disorders: a systematic review and meta-analysis Molecular Psychiatry 17:290–314
  • Rubenstein et al. 2015. A review of the differences in developmental, psychiatric, and medical endophenotypes between males and females with autism spectrum disorder. J Dev Phys Disabil 27:119–139
  • Russo et al. 2018. Modeling the Interplay Between Neurons and Astrocytes in Autism Using Human Induced Pluripotent Stem Cells. Biological Psychiatry 83:569–578.
  • Rutter. 2005. Incidence of autism spectrum disorders: changes over time and their meaning. Acta Paediatr 94:2–15
  • Sajdel-Sulkowska et al. 2008. Oxidative stress in autism: elevated cerebellar 3-nitrotyrosine levels. Am J Biochem Biotechnol 4:73–84
  • Schechter and Grether. 2008. Continuing increases in autism reported to California’s developmental services system: mercury in retrograde. JAMA Physch 65:19–24
  • Schulz. 2000. Glutathione, oxidative stress and neurodegeneration. Eur J Biochem. 267:4904–4911
  • Scott et al. 2017. Clinical clues for autoimmunity and neuroinflammation in patients with autistic regression. Developmental Medicine and Child Neurology 59:947–951.
  • Seneff et al. 2012. Empirical data confirm autism symptoms related to aluminum and acetaminophen exposure. Entropy 4:2227–2253
  • Shandley and Austin. 2011. Ancestry of pink disease (infantile acrodynia) identified as a risk factor for autism spectrum disorders. J Toxicol Environ Health A 74:1185–1194
  • Sharpe et al. 2012. Thimerosal-derived ethylmercury is a mitochondrial toxin in human astrocytes: possible role of fenton chemistry in the oxidation and breakage of mtDNA. J Toxicol 373678
  • Sharpe et al. 2013. B-Lymphocytes from a population of children with autism spectrum disorder and their unaffected siblings exhibit hypersensitivity to thimerosal. J Toxicol 2013:801517
  • Shaw and Tomljenovic. 2013a. Administration of aluminium to neonatal mice in vaccine-relevant amounts is associated with adverse long term neurological outcomes. J Inorg Biochem 128:237–244
  • Shaw and Tomljenovic. 2013b. Aluminum in the central nervous system (CNS): toxicity in humans and animals, vaccine adjuvants, and autoimmunity. Immun Res 56:304–316
  • Shaw et al. 2014a. Aluminum-induced entropy in biological systems: Implications for neurological disease. J Toxicol 2014:491316
  • Shaw et al. 2014b. Etiology of autism spectrum disorders: Genes, environment, or both? OA Autism 2:11
  • Sienkiewicx et al. 2012. Neurological adverse events following vaccination. Prog health Sci 2:129–141
  • Singh. 2009. Phenotypic expression of autoimmune autistic disorder (AAD): A major subset of autism. Ann Clin Psychiat 21:148–161
  • Singh et al. 1998. Serological association of measles virus and human herpesvirus-6 with brain autoantibodies in Autism. Clin Immunol Immunopathol 89:105–108
  • Singh et al. 2002. Abnormal measles-mumps rubella antibodies and CNS autoimmunity in children with autism. J Biomed Sci 9:359–364
  • Smeeth et al. 2004. MMR vaccination and pervasive developmental disorders: a case-control study. Lancet 364:963–969
  • Stratton et al. (eds). 2011. Adverse Effects of Vaccines: Evidence and Causality. Committee to Review Adverse Effects of Vaccines, Institute of Medicine. Washington (DC): National Academies Press (US)
  • Sun et al. 2018. Early seizures prematurely unsilenced auditory synapses to disrupt thalamocortical critical period plasticity. Cell Reports 23:2533–2540
  • Richler et al. 2006. Is there a ‘regressive phenotype’ of autism spectrum disorder associated with measles-mumps-rubella vaccine? A CPEA study. J Autism Dev Disord 36:299–316
  • Ryu et al. 2017. Associations of prenatal and early childhood mercury exposure with autistic behaviors at 5 years of age: The Mothers and Children’s Environmental Health (MOCEH) study. Science of the Total Environment 605-606: 257.
  • Stajich et al. 2000. Iatrogenic exposure to mercury after hepatitis B vaccination in preterm infants. J Pediatr 136:676–681
  • Stamova et al. 2009. Correlations between gene expression and mercury levels in blood of boys with and without autism. Neurotox Res 19:31–48
  • Sulkowski et al. 2012. Maternal thimerosal exposure results in aberrant cerebellar oxidative stress, thyroid hormone metabolism, and motor behavior in rat pups; sex- and strain-dependent effects. Cerebellum 11:575–586
  • Taylor et al. 1999. Autism and measles, mumps, and rubella vaccine: no epidemiological evidence for a causal association. Lancet 353: 2026–2029
  • Taylor. 2006. Vaccines and the changing epidemiology of autism. Child Care Health Dev. 32:511–519
  • Taylor et al. 2014. Vaccines are not associated with autism: and evidence-based meta-analysis of case-control and cohort studies. Elsevier 32:3623-3629
  • Tomljenovic and Shaw. 2011. Do aluminum vaccine adjuvants contribute to the rising prevalence of autism? J Inorg Biochem 105:1489–1499
  • Tomljenovic and Shaw. 2012. Mechanisms of aluminum adjuvant toxicity and autoimmunity in pediatric populations. Lupus 21:223–230
  • Tsumiyama et al. 2009. Self-organized criticality theory of autoimmunity. PLoS ONE 4:e8382
  • Uno et al. 2015. Early exposure to the combined measles-mumps-rubella vaccine and thimerosal-containing vaccines and risk of autism spectrum disorder. Vaccine 33:2511–2516
  • Uchiyama et al. 2007. MMR-vaccine and regression in autism spectrum disorders: negative results presented from Japan. J Autism Dev Disord 37:210–217
  • Vargas. 2005. Neuroglial activation and neuroinflammation in the brain of patients with autism. Ann Neurol 57:67–81
  • Vasquez 2017. Biological plausibility of the gut-brain axis in autism. Annals of the New York Academy of Sciences 1408:5–6.
  • Verstraeten et al. 2003. Safety of Thimerosal-Containing Vaccines: A two-phased study of computerized health maintenance organization databases. Pediatrics 112:1039–1048
  • Verstraeten et al. 2014. Increased risk of developmental neurologic impairment after high exposure to thimerosal-containing vaccine in first month of life. Proceedings of the Epidemic Intelligence Service Annual Conference 49
  • Vestergaard et al. 2004. MMR vaccination and febrile seizures: evaluation of susceptible subgroups and long-term prognosis. JAMA 292:351–357
  • Waly et al. 2004. Activation of methionine synthase by insulin-like growth factor-1 and dopamine: a target for neurodevelopmental toxins and thimerosal. Mol Psychiatry 9:358–70
  • Walker et al. 2006. Cultured lymphocytes from autistic children and non-autistic siblings up-regulate heat shock protein RNA in response to thimerosal challenge. Neurotoxicology 27:685–692
  • Walker et al. 2013. Identification of unique gene expression profile in children with regressive autism spectrum disorder (ASD) and ileocolitis. PLoS ONE 8: e58058
  • Wang et al. 2018. IL-4 mediates the delayed neurobehavioral impairments induced byneonatal hepatitis B vaccination that involves the down-regulation of the IL-4 receptor in the hippocampus. Cytokine 110:137–149.
  • Weizman et al. 1982. Abnormal immune response to brain tissue antigen in the syndrome of autism. Am J Psychiatry 139:1462-1465
  • Werner and Dawson. 2005. Validation of the phenomenon of autistic regression using home videotapes. Arch Gen Psychiatry 62:889–895
  • Wilson et al. 2011. Adverse events following 12 and 18 month vaccinations: a population-based, self-controlled case series analysis. PLoS ONE 6:e27897
  • Windham et al. 2006. Autism spectrum disorders in relation to distribution of hazardous air pollutants in the San Francisco bay area. Environ Health Perspect 114:1438–1444
  • Woods et al. 2010. Urinary porphyrin excretion in neurotypical and autistic children. Environ Health Perspect 118:1450–1457
  • Wong et al. 2014. Methylomic analysis of monozygotic twins discordant for autism spectrum disorder and related behavioural traits. Molecular Psychiatry 19:495–503.
  • Wu et al. 2008. Thiol-modulated mechanisms of the cytotoxicity of thimerosal and inhibition of DNA topoisomerase II alpha. Chem Res Toxicol 21:483–493
  • Yasuda and Tsutsui. 2013. Assessment of infantile mineral imbalances in autism spectrum disorders (ASDs). Int J Environ Res Public Health 10:6027–6043
  • Yassa. 2014. Autism: a form of lead and mercury toxicity. Environ Toxicol Pharmacol. 38:1016–1024
  • Yel. 2005. Thimerosal induces neuronal cell apoptosis by causing cytochrome c and apoptosis-inducing factor release from mitochondria. Int J Mol Med 16:971–977
  • Young et al. 2008. Thimerosal exposure in infants and neurodevelopmental disorders: an assessment of computerized medical records in the Vaccine Safety Datalink. J Neurol Sci 271:110–118
  • Young et al. 2010. Porphyrinuria in Korean children with autism: correlation with oxidative stress 73:701–710
  • Young et al. 2011. Aberrant NF-KappaB expression in autism spectrum condition: A mechanism for neuroinflammation. Front Psychiatry 2:27
  • Zhang et al. 2012. Risk factors for autistic regression: results of an ambispective cohort study. J Child Neurol. 27:975–981
  • Zhang et al. 2014. Thioredoxin: A novel, independent diagnosis marker in children with autism. Int J Dev Neurosci 40:92–66
Posted in Vaccines/Alternative Medicine | Tagged , , , , , , , , | 4 Comments

How not to science: Lessons from flat earthers and climate change deniers

A great quote from one of my favorite shows, Stargate SG1 (Season 3 Episode 19, “New Ground”)

Science is an amazingly powerful tool for disentangling fact and fiction. When done correctly, it is a systematic, objective, unbiased, and self-correcting method for understanding our universe. Unfortunately, many people don’t appreciate the objectivity that science requires, and instead view it as a blunt instrument for proving what they already “know” to be true. You see, science always has to ask open-ended questions, because science is not a method for trying to prove something. Rather, science is a method for trying to learn what is true. It is a method for setting aside your biases, testing possibilities, and discerning objective reality. Thus, science must always go from evidence to a conclusion. It can never go from a conclusion to evidence. Indeed, if you start with a conclusion, then look for evidence to support that conclusion, you are, by definition, doing pseudoscience. This fraudulent strategy of trying to prove a belief while operating under the guise of science is extremely common among science deniers, and in this post, I want to use two recent, widely reported events as illustrations of how this flawed line of reasoning operates and why it is problematic.

The events in question are the Netflix documentary about flat earthers and the announcement that the Trump administration is putting together an “adversarial” climate change panel. Although I’m going to focus on just these two examples, I want to make it clear that the underlying cognitive biases and abuses of science that I’m talking about are pervasive throughout pseudoscience. Anti-vaccers, creationists, anti-GMO activists, proponents of alternative medicines, etc. all do this.

Let’s start with flat earthers. Netflix recently released its documentary, “Behind the Curve” that takes a look at flat earthers. There are several really revealing moments, but I want to focus on the gyroscope experiment. You see, some of the flat earthers conducted an experiment that actually showed that the earth is round, but as you might have guessed, they were not eager to embrace the results of their own experiment.

In the lead up to this experiment, Jeran Campanella (one of the people who runs the “Globebusters” Youtube channel) stated, “I think that the scientific method is the best way to get to the truth.” So far so good, but he ended that sentence with, “and I just want to feel comfortable in things that I believe.” Now we’ve entered problematic territory. Science is not a method for making you feel comfortable. It is a method for discerning what is true, regardless of whether the truth makes you comfortable. To be fair, perhaps what Jeran meant was simply that he wanted to accept whatever position was supported by good evidence so that he could be comfortable in his views, but the response to the gyroscope experiment suggests otherwise.

The experiment itself was fairly simple. On a spinning globe, a gyroscope at any fixed point on the planet will drift (on earth, this drift should be 15° an hour due to the fact that the earth makes a full rotation every 24 hours). So, the flat earthers bought a $20,000 gyroscope to “test” this. This is actually a good experiment. It presents a nice falsifiable prediction (the heart and soul of science). If the earth is round and is rotating, the gyroscope will not only drift, but it will do so at 15° an hour. If the earth is flat, the gyroscope will not drift. Nevertheless, I put the word “test” in quotes a minute ago, because the flat earthers weren’t actually interested in testing anything. They were interest in proving that the earth is flat rather than objectively testing its shape. I say this, because, as you should have expected, the gyroscope did, in fact, drift by 15 degrees an hour. Thus, the notion of a flat earth was falsified, but here is how Bob Knodel (Jeran’s co-host on Globebusters) responded to the experiment,

“Wow, that’s kind of a problem, right?” Yes, Bob, it is. It defeats your hypothesis, but he continued, “We, obviously, were not willing to accept that.”

Here we have the key problem. You don’t get to ignore a result just because you don’t like it. That’s not how science works, and it is why it is so problematic to start with a conclusion, rather than starting with evidence. Flat earthers will never accept evidence against their view (as Bob just admitted). They will always ignore evidence to the contrary and cherry-pick and distort evidence as much as they have to in order to maintain their beliefs. If you start with a conclusion, you will always be able to find evidence which, at least to you, appears to support your conclusion. This is a very easy cognitive trap to fall into, and it is why science must always start with the evidence, then draw a conclusion based on that evidence, regardless of what the evidence shows.

Back briefly to the example, Bob continued, “We started looking for ways to disprove that it [the gyroscope] was actually registering the motion of the earth.” Again, this is not how science works. Science must always ask open-ended questions and if your goal is to prove or disprove something, then you are doing pseudoscience. In this particular case, the flat earthers decide that the drift must be from energy from the heavens. This brings me to another important point. When faced with contrary evidence, science deniers often simply make things up to patch holes in their arguments. In this case, there is no evidence for these heavenly energies, but flat earthers assume that they exist because they need them to explain various phenomena. In technical terms, this type of blind assumption is known as an ad hoc fallacy.

Moving on, the flat earthers put the gyroscope in a zero gauss chamber to shield it from these “heavenly forces,” but that test still showed the 15° drift that you expect from a spinning sphere, a result that Bob described as “unfortunate.” Again, you shouldn’t think that it is “unfortunate” if a test disproves you. You should embrace the knowledge that the test provided, regardless of how it aligns with your previous views. Bob, of course, is unwilling to do that, and plans on running future tests to try to prove the gyroscope isn’t actually showing evidence of a spinning earth.

Hopefully, at this point, you can see why this approach to “science” is so problematic, but as I said earlier, this is not unique to flat earthers. Indeed, in previous posts, I’ve argued that all forms of science denial are fundamentally the same, and the flawed reasoning used by flat earthers is the same flawed reasoning that is used by people such as climate change deniers.

This brings me to the announcement that the White House is putting together an “adversarial” climate change panel that will be tasked with discrediting various aspects of climate science. The impetus for this seems to be the release of the Fourth National Climate Assessment. This government review of the evidence found that climate change is being cause by us and is a serious problem. Trump and other climate change deniers obviously did not like that answer, so they are putting together this “adversarial” panel to disprove the evidence.

that's not how this works memeThis is exactly the same thing that flat earthers do, and its not how science works. The reality is that the evidence for climate change is overwhelming. There is an extremely strong consensus among studies, and studies to the contrary are virtually non-existent. Systematic reviews of the literature consistently find that we are causing the climate to change, but, just like flat earthers, climate change deniers don’t want to accept that evidence. Rather, they want to form a group of “experts” to cherry-pick evidence and try to prove a particular position. That is, by definition, pseudoscience. Science works precisely because it is objective and asks open questions rather than blindly trying to prove a particular position. If you start with the goal of trying to discredit climate change or vaccines or any other position, you will always see only what you want to see, now matter how wrong you are. You will always be able to convince yourself that the evidence is on your side. Getting a bunch of people who agree with you together and ordering them to discredit a position is simply not how science works.

In closing, I want to reiterate that nothing that I have talked about is unique to flat earthers or climate change deniers. As I’ve talked about before, anti-vaccers love to cherry-pick lists of papers that supposedly support their position, while blindly ignoring the mountain of studies that disagree with them. Tenpenny (a prominent anti-vaccer) has even gone as far as putting together a “library” with the expressed purpose of giving people, “evidence to support what they intuitively know” (details here). Similarly, young earth creationists have groups like the Institute for Creation Research that try to prove creationism “scientifically.” The IARC report that claimed glyphosate is carcinogenic is another great example that parallels Trump’s climate change panel. An examination of that report quickly reveals that it was heavily biased and cherry-picked its sources to try to prove that glyphosate was carcinogenic, while totally ignoring all the sources to the contrary (more details here and here).

My point is that this type of motivated reasoning can be found on just about any topic, and you need to be wary of it. When you approach a topic, you need to make sure that you are asking open questions and accepting the answers to those questions, rather than trying to prove a position. Always ask yourself, “what evidence would convince me that I’m wrong?” If the answer is, “nothing,” then you are not adhering to the rules of science or logic.

Related posts                                                                                                           

Posted in Global Warming, GMO, Nature of Science, Science of Evolution, Vaccines/Alternative Medicine | Tagged , , , , , , , | 12 Comments

5 hottest years on record all happened in the past 5 years (global warming is real)

Climate change is based on scientific facts and evidence, not politics or ideology, and it is an incontrovertible fact that the planet is warming. Nevertheless, many people continue to deny this reality (although polls suggest that their numbers are shrinking). The recent polar vortex, for example, was frequently touted as evidence that global warming is not happening. The actual data, of course, paint a very different picture. NASA has just released its final 2018 data for its comprehensive global Land-Ocean Temperature Data set, and the data very clearly show that the planet is on a warming trajectory. Indeed, all five of the five hottest years on record have occurred in the past five years. Sure, there have been cold spells and winter storms during those years, but those are weather events, not climate trends. Indeed, climate change never predicted that there wouldn’t be cold winters or blizzards. Rather, the prediction always has been that the mean temperature will increase, along with other changes like increased floods in some areas, increased droughts in others, etc. (all of which are coming true).

Global average annual temperature anomalies from NASA’s global Land-Ocean Temperature Index data set (1951-1980 base period). I highlighted the 5 hottest years in red and 5 coldest years in blue.

If you are someone who doubts that the climate is actually warming, then I implore you to actually look at the data. Just look at it. The warming trend is so obvious. The 20 hottest years on record have all occurred within the past 22 years! That’s incredible. If the planet wasn’t actually warming, then a trend like that should not exist. We would expect really cold years and really hot years to be interspersed together, but what we actually see is all the really cold years clumped together a century ago and all the really hot years clumped together over the past two decades.

global warming cliamte change data history

Global average annual temperature anomalies from NASA’s global Land-Ocean Temperature Index data set (1951-1980 base period). I highlighted the 20 hottest years in red and 20 coldest years in blue.

Further, this pattern exists across data sets. For example, the HadCRUT4 dataset from the Hadley Center also shows that the past five years have been the five hottest on record. Some other data sets disagree slightly (usually with a year other than 2014 for the 5th hottest), but the overarching pattern is there, and the warming is undeniable.

But what about this notion of a global warming pause or hiatus? I wrote about that at length several years ago and explained that the only way to get a “pause” was to cherry-pick the data  set, cherry-pick the starting year, and ignore confounding factors, all of which is statistically invalid. You see, satellite measurements tend to be quite sensitive to factors like El Niños. So, to create the illusion of a pause, charlatans cherry-picked a satellite data set (the RSS/MSU dataset), then cherry-picked a starting year with a huge El Niño effect (1998). Thus, by cherry-picking an unreasonable and biased starting point, they could mask the real trend. Indeed, when you account for El Niños, even those cherry-picked dates and data sets show warming (Foster and Rahmstorf 2011). Multiple studies have looked at this and consistently found that there is no pause in climate change (Easterling and Wehner 2009; Santer et al. 2011; Lewandowsky et al. 2015a; Lewandowsky et al. 2015b; Lewandowsky et al. 2018).

Data from the RSS/MSU satellite data set (TTT). Notice the massive El Niño spike at 1998. Climate change deniers like to cherry pick that as the starting point for their claim that global warming has paused. When you look at the full data set, hopefully you can see why that is invalid cherry-picking. The full data set clearly shows a warming trend. These data are monthly.

Additionally, now that several years have passed since I last wrote about the pause, let’s take a look at where things stand now. Using the TTT MSU data set and 1998 as our starting year (as deniers like to do), and using only the average temperature per year, we find a positive trend with a P value of 0.066. In simplest terms, a P value is a probability that a result as great or greater than the observed result could arise by chance, and we typically determine significance with a largely arbitrary threshold of 0.05. Thus, although this trend is not statistically significant, it is pretty dang close with only a 6.6% chance that a result like that could arise by chance (many, like me, argue that 0.05 should not be treated as a magical threshold, but that is a debate for another time). Further, I need to point out again that this lack of significance is entirely because we cheated and cherry-picked our starting year. If we start with the data in 1997 instead of 1998, the trend is significant with P = 0.028. Similarly, starting in 1999 produces a significant result (P = 0.005). In other words, if you are going to try to cherry-pick dates and argue that there hasn’t been a significant warming trend since 1998, you also have to acknowledge that there has been a significant warming trend since 1997 as well as since 1999. Do you see why cherry-picking the starting year is ridiculous now? Finally, it is worth mentioning that the barely non-significant result in 1998 is also partially due to a lack of sample size. If you use the monthly data instead of the annual means, then even starting in 1998 you get a strongly significant warming trend (P < 0.001).

There are two strategies left that people use to try to get out of the fact that the climate is changing. The first is to say that the recent warming has all been from El Niños. There certainly have been strong El Niño effects at various points, but they cause clusters of years to spike, rather than the type of decades-long trend we are in. Further, as stated earlier, removing the El Niño effect from the data sets makes the warming trend stronger, not weaker (Foster and Rahmstorf 2011).

The second strategy is to argue that the models have all been wrong. This claim is false. Sure, you can find graphs online that purportedly show that the models have been wrong, but they all use the same type of dishonest techniques that I have been describing throughout this post. Actual scientific analyses that used proper statistical methods have consistently shown that the models have been accurate (Hansen et al. 2006; Frame and Stone 2012; Rahmstorf et al. 2012; Cowtan et al. 2015; Marotzke and Firster 2015; Lewandowsky et al. 2018). More details here.

In short, the planet is warming. This is an empirical fact that is borne out by every data set available. Anyone who claims otherwise is denying evidence. This is not a “hoax” or “liberal propaganda.” It is a fact, and you don’t have to take my word for it. Just look at the data. Just look at it. The trend is obvious. When you hear people make claims like, “the scientist’s predictions have all been wrong” or ask things like, “where is global warming?” they are displaying a willful ignorance of reality.

In closing, to anyone who doesn’t believe in global warming, I would like to know what it is going to take to convince you. What evidence would convince you that you are wrong? The arguments against climate change have been the same for decades, yet year after year more data is collected showing that climate change is real and caused by us, more temperature records are broken, more extreme weather events happen, etc. At what point are you going to be willing to accept that you are wrong?

Note: Although this is beyond the topic of this post, I will point out that we have extremely compelling evidence from thousands of studies showing that we are the cause behind this warming and it is already having dire consequences (details and sources here and here).

Related posts

Literature Cited

  • Cowtan et al. 2015. Robust comparison of climate models with observations using blended land air and ocean sea surface temperatures. Geophysical Research Letters 42:6526–6534.
  • Easterling and Wehner 2009. Is the climate warming or cooling? Geophysical Research Letters 36.
  • Foster and Rahmstorf 2011. Global temperature evolution 1979–2010. Environmental Research Letters 7:011002.
  • Frame and Stone 2012. Assessment of the first consensus prediction on climate change. Nature Climate Change 3:357–359.
  • Hansen et al. 2006. Global temperature change. Proceedings of the National Academy of Sciences 10314288–14293.
  • Lewandowsky et al. 2015a. On the definition and identifiability of the alleged hiatus in global warming. Scientific Reports 5: 16784.
  • Lewandowsky et al. 2015b. The “pause” in global warming: Turning a routine fluctuation into a problem for science. Bulletin of the American Meteorological Society 96:723–733.
  • Lewandowsky et al. 2018. The ‘pause’ in global warming in historical context: (II). Comparing models to observations. Environmental Research Letters
  • Marotzke and Firster 2015. Forcing, feedback and internal variability in global temperature trends. Nature 517:565–570.
  • Santer et al. 2011. Separating signal and noise in atmospheric temperature changes: The importance of timescale. Journal of Geophysical Research: Atmospheres 116.
  • Rahmstorf et al. 2012. Comparing climate projections to observations up to 2011. Environmental Research Letters 7:044035.
Posted in Global Warming | Tagged , | 1 Comment

Do Bt GMOs “make their own poison”? Only if you’re an insect

I frequently write about genetic engineering (GE) and genetically modified organisms (GMOs) on this blog, and I do that because GMOs are often misunderstood and villainized when, in reality, they have enormous benefits and a huge potential both for human health and protecting the environment. We can design them to have increased nutritional content (e.g., golden rice), to brown more slowly thus resulting in less food waste (e.g., arctic apples), to have reduced carcinogens thus lowering cancer risk (e.g., GMO potatoes), to be herbicide resistant thus increasing farming efficiency and reducing land use (e.g., Roundup-ready crops), to be resistant to insects thus reducing pesticide use (e.g., Bt GMOs), etc. Yet despite all these benefits, ill-founded arguments against GMOs abound.

In this post, I specifically want to talk about the Bt GMOs, because they are particularly beneficial. I’ve written about their benefits at length before, but, in short, because they produce their own insecticides rather than having to be sprayed with insecticides, implementing them results in a massive reduction in pesticide use (Shelton et al. 2002; Cattaneo 2006; Lu and Desneux 2012), increased crop yields (Shelton et al. 2002; Cattaneo 2006; Vitale et al. 2010), reduced impacts on non-target organisms (e.g., bees and butterflies; Marvier et al. 2007; Wolfenbarger et al. 2008; Comas et al. 2014), improved bio-diversity (Lu and Desneux 2012), and enormous economic benefits for farmers (Hutchison et al. 2010). Given all these advantages when compared to either traditional or organic farming, you’d think everyone would be onboard with Bt GMOs. After all, by far, the three most common anti-GMO arguments that I encounter are 1). GMOs increase pesticides, 2). GMOs are bad for biodiversity/bees/monarchs, and 3). GMOs are somehow bad for farmers. These arguments are problematic in general, but they are clearly blatantly false when it comes Bt GMOs. Nevertheless, Bt GMOs are villainized, often by arguing that they “obviously must be bad for human health since they produce their own poison.” This argument generally commits both an appeal to emotion fallacy and a common sense fallacy, but it is also indefensible scientifically, so I want to explain why it is a nonsense argument.

Note: Insecticides are simply  pesticides that target insects (herbicides are pesticides that target weeds, fungicides are pesticides that target fungi, etc.)

What is Bt toxin and how is it used?

To being understanding this topic, we need to take a step back and talk more generally about what Bt toxin is and how we have used it historically. It is a chemical (technically a group of chemicals) that is naturally produced by the bacterium Bacillus thuringiensis, and it is strongly insecticidal (i.e., it kills insects). When designing Bt GMOs, scientists took advantage of those insecticidal traits and modified the genes of various crops so that they too would produce Bt toxin. Thus, by expressing those genes, Bt GMOs can produce Bt toxin and kill any insects that try to eat them.

Using Bt toxin to control pests is, however, hardly new. It has been sprayed on crops as an insecticide for decades, including widespread use in organic farming (yes, organic farming does use pesticides, it just limits itself to “natural” pesticides). In other words, this is not something novel that was developed for GMOs. The chemical has been commonly used for decades and is extremely safe for humans (I’ll talk about why it is safe in a minute; Mendelsohn et al. 2003). Indeed, in some parts of the world it is even added to drinking water reservoirs to reduce the larvae of harmful insects like mosquitoes, and this is considered to be safe by the World Health Organization (WHO/IPCS. 1999). Further, its widespread use in traditional agriculture (especially organic agriculture) means that you are already exposed to this chemical in your food on a regular basis, and that is fine, because it’s safe for humans at anything but an insane dose. Indeed, its use is so prevalent, that one paper (Hammond and Koch 2012) concluded that,

“It seems likely that dietary exposure to functionally active Cry proteins from application of Bt microbial formulations to vegetables (shortly before harvest) could be similar to or even higher than dietary exposure from consumption of foods derived from Bt crops.”

In other words, you receive a similar exposure to Bt regardless of whether you eat GMOs or conventional/organic crops.

I wanted to take the time to go through all this to make it clear that Bt GMOs aren’t producing some mad science chemical that was concocted in a lab (not that such a chemical would automatically be dangerous). Rather, they are making use of a chemical that is already widely used and that you are already constantly exposed to. The only difference is that for regular crops, the chemical has to be sprayed on entire fields, which wastes time, money, and water, increases greenhouse gas emissions, and kills any non-target insects in the field. In contrast, Bt GMOs produce the chemical themselves, which means that farmers don’t need to use nearly as many pesticides and only insects that actually eat the crops (i.e., pests) are affected. All of that is obviously a huge advantage for Bt GMOs both environmentally and economically.

Bt is safe

At this point, you may legitimately be wondering how Bt toxin can simultaneously be lethal to insects at a very low dose, but safe for humans and anything but an unreasonably high dose. The answer is something known as host specificity. All animals are biochemical machines that run via chemical reactions, and “toxins” or “poisons” operate by either impeding reactions that should be happening or causing reactions that shouldn’t be happening. However, different groups of animals have different biochemistry, and the biochemistry of a mammal (like you and me) is quite different from the biochemistry of an insect. As a result, the chemical mechanisms through which Bt kills insects don’t occur in humans. To put that another way, the fact that something is a poison to insects doesn’t automatically mean that it is a poison to humans.

Without getting too technical, when Bt enters an insect’s digestive system, the alkaline environment causes it to release “Cry proteins.” These proteins bind chemically to specific receptors on the lining of the insect’s gut, which sets off a series of chemical reactions, ultimately resulting in the death of the insect. All of this is very specific to insects and simply cannot happen in humans. First, our stomachs are acidic, not alkaline. So, our stomachs largely degrade the Cry proteins rather than releasing them (Cao et al. 2010). Further, the Cry proteins that survive digestion still can’t do anything harmful because the guts of mammals lack the specific Cry protein-binding receptors that are found in insects (Noteborn et al. 1995). Additionally, even if the Cry proteins manage to bind with non-specific sites, the bond is generally weak and does not cause the damaging chemical reactions that occur in insects (Shimada et al. 2006).

All of this simply means that Bt is very safe in humans because the mode of action through which it kills insects can’t occur in humans. In other words, Bt has a high host specificity (i.e. it very specifically targets insects while being safe for mammals). Thus, when talking about humans, it is incorrect to say that Bt GMOs “make their own poison,” because Bt is not poisonous to humans (except at a ridiculous hypothetical dose). Indeed, we consume many things that are poisonous to other animals but safe for us (e.g., chocolate is poisonous to dogs, avocados are poisonous to parrots, etc.).

The conclusion that Bt is safe for humans has, of course, been borne out by numerous studies (reviewed in Hammond and Koch 2012; Hammond and Cockburn 2008). As is standard for toxicology studies, many of these used rat and mice models because our physiology is actually very similar to that of mice and rats, making them good models for understanding toxicity. These studies used crazy high doses (often 4000 mg/kg/day or more) that are way above what you would ever receive from eating GMO crops, and they still failed to find any adverse effects. To get that type of dose in your food from a Bt crop, you would have to eat several hundred thousand kilograms of produce a day (Hammond and Koch 2012)! It is, quite simply, not possible for you to eat enough vegetables to get anything even close to a dangerous dose of Bt. Even Jabba the Hutt on a vegetarian diet couldn’t achieve a toxic dose from eating Bt GMOs. Also, keep in mind, those absurd doses weren’t for LD50s; they were being used to look for any adverse effects and failed to find any. So the dose that actually starts to cause problems in humans is even higher than the already absurd doses being used in those studies.

Note: I’m assuming that Jabba the Hutt also lacks Cry protein-binding receptors. I’m not sure if that’s canon.

At this point, it should be obvious that the production of Bt toxin by Bt GMOs does not pose a health risk, because the toxicity of Bt to humans is so incredibly low. Nevertheless, multiple studies have looked at the effects of consuming Bt GMOs (rather than just Bt toxin), and, as you might have guessed, they did not find any evidence that that Bt GMOs are less nutritious than conventional crops, nor did they find any evidence that consuming Bt GMOs is harmful (even when consumed in large quantities, daily, for months at a time; Flachowsky et al. 2007; McNaughton et al. 2007; Schrøder et al. 2007; Scheideler et al. 2008; Yuan et al. 2013). Indeed, there is evidence that Bt GMOs are actually safer than their conventional/organic alternatives, because Bt GMOs have reduced mycotoxins (these come from fungi that like to live in holes made by insects; Pellegrino et al. 2018).

What about the microbiome?

At this point, I often find that people resort to speculation that the consumption of Bt could disrupt the gut microbiome. First, this argument is not exclusive to GMOs since, as explained earlier, you get Bt toxin for conventional and organic crops as well. Second, multiple studies have looked at the effects of Bt GMO consumption on the gut microbiome, and the results range from “no effect” to (paraphrasing) “slight effect that doesn’t appear to be harmful” (Einspanier et al. 2004; Wiedemann et al. 2007; Buzoianu, et al. 2012, 2013; Yuan et al. 2013). Keep in mind, just about everything affects the microbiome to some degree. So, the relevant question is not, “does it shift the microbiome” but rather, “does it shift it in a way that is harmful?” The evidence to date says that the answer to that question is, “no.”

What about gene transfer?

Another counter argument is that the real danger is that horizontal gene transfer will take place and the genes for Bt will end up either in our genome or the genome of some gut bacteria (horizontal gene transfer is where one organism incorporates another organism’s DNA into its own genome; it is basically nature making a transgenic GMO). This is a concern that I frequently hear about GMOs, and it is unmerited for numerous reasons. First, your digestive system does a pretty good job of ripping apart and degrading DNA. It’s not likely that entire genes will make it past the stomach (Rizzi, A. et al. 2012; Yuan et al. 2013). Second, even if they did, why should the one or two genes that we put into a GMO be more concerning than the millions of genes in everything you eat!? When you eat an apple, you ingest the entire genome of an apple (which includes things like genes for producing cyanide); yet I have never heard anyone express concern over horizontal gene transfer from an apple. So, this argument is completely disingenuous. Why should you be concerned about the genes for Bt in a GMO but not concerned about the genes for cyanide in an apple?

Further, Bacillus thuringiensis (the bacteria that we got the Bt genes from) is an extremely common environmental bacteria. I guarantee you that you have eaten that bacteria numerous times in your life. Further, other bacteria that you have ingested have surely been exposed to Bacillus thuringiensis before being ingested. In other words, there are plenty of opportunities for Bt horizontal gene transfer without GMOs, yet no one is concerned about them (with good reason). Like so many anti-GMO arguments, this line of reasoning holds GMOs to an extremely different standard than everything else. It’s just not a legitimate concern.

“But I don’t want poisons being produced inside me”

I’ve saved this one for last because it is, quite frankly, silly. Nevertheless, I do encounter it from time to time, so let’s talk about it. This argument asserts that what I have said about doses is wrong, because the crops will continue to produce Bt toxin while inside you (thus creating high doses). The problem with this is obviously that vegetables are not continuing to perform biological processes in your gut. They are dead, chewed up, and digested. This argument is like being worried that swallowing a seed will cause a tree to grow out of your stomach.

Conclusion

The argument that Bt GMOs are dangerous because they “make their own poison” is a nonsense argument. It appeals to emotions and the notion of common sense, rather than scientific evidence or logic. The reality is that Bt toxin is host specific, so while it is fatal to insects at even a very low dose, it is safe for humans at even an extremely high dose. Further, it has been used as a pesticide in both conventional and organic farming for decades, and the amount of Bt you are exposed to from Bt GMOs is very similar to the amount you are exposed to from regular crops. Bt GMOs have been extensively studied and are extremely safe. Even if you ate thousands of kilograms of produce a day, you still would not receive anything even close to a dangerous dose of Bt.

Finally, you don’t have to take my word for this. There is an extremely strong consensus on this topic in the scientific literature, and I have only cited a tiny handful of the available studies. Here are some literature reviews that are worth reading: WHO/IPCS. 1999, Betz et al. 2000, OECD. 2007, Hammond and Koch. 2012, Koch et al. 2015.

 Related posts

Suggested further reading

 Literature cited

  • Betz et al. 2000. Safety and advantages of Bacillus thuringiensis-protect-ed plants to control insect pests. Regulatory Toxicology and Pharmacology 32:156 –173
  • Buzoianu, et al. 2012. The effect of feeding Bt MON810 maize to pigs for 110 days on intestinal microbiota. PLoS ONE 7:e33668
  • Buzoianu et al. 2013. Sequence-based analysis of the intestinal microbiota of sows and their offspring fed genetically modified Bt maize in a trans-generational study. Applied Environmental Microbiology
  • Cao et al. 2010. Safety assessment of Cry1C protein from genetically modified rice according to the national standards of PR China for a new food resource. Regulatory Toxicology and Pharmacology 58:474–481.
  • Cattaneo 2006. Farm-scale evaluation of the impacts of transgenic cotton on biodiversity, pesticide use, and yield. Proceedings of the Royal Academy of Sciences 103:7571–7576
  • Comas et al. 2014. No effects of Bacillus thuringiensis maize on nontarget organisms in the field in southern Europe: a meta-analysis of 26 arthropod taxa. Transgenic Research 23:135–143
  • Einspanier et al. 2004. Tracing residual recombinant feed molecules during digestion and rumen bacterial diversity in cattle fed transgene maize. European Food Research and Technology 218:269-273
  • Flachowsky et al. 2007. Studies on feeds from genetically modified plants (GMP)—Contributions to nutritional and safety assessment. Animal Feed Science and Technology 133:2–30
  • Hammond and Cockburn. 2008. The safety assessment of proteins introduced into crops developed through agricultural biotechnology: a consolidated approach to meet current and future needs. In: Hammond BG (ed) Food safety of proteins in agricultural biotechnology. CRC Press, New York
  • Hammond and Koch. 2012. A review of the food safety of Bt p305–325 in Bacillus thuringiensis Biotechnology. (Sansinenea ed.), Springer
  • Koch et al. 2015. The food and environmental safety of Bt Frontiers in Plant Science 6:283
  • McNaughton et al. 2007. Feeding performance in broiler chickens fed diets containing DAS-59122-7 maize grain compared to diets containing non-transgenic maize grain. Animal Feed Science and Technology 132:227–239.
  • Lu and Desneux 2012. Widespread adoption of Bt cotton and insecticide decrease promotes biocontrol services. Nature 487:362–365
  • Marvier et al. 2007. A meta-analysis of effects of Bt cotton and maize on nontarget invertebrates. Science 316:1475–1477
  • Mendelsohn et al. 2003. Are Bt crops safe? Nature Biotechnology 21:1003–1009
  • Noteborn et al. 1995. Safety assessment of the Bacillus thuringiensis insecticidal crystal protein CRYIA(b) expressed in tomato. p134­–147 In: Genetically Modified Foods: Safety Issues. Engle et al. eds. American Chemical Society, Washington, DC.
  • 2007. Consensus document on safety information on transgenic plants expressing Bacillus thuringiensis—derived insect control proteins. In Joint Meeting of the Chemicals Committee and the Working Party on Chemicals, Pesticides and Biotechnology, Paris, France
  • Pellegrino et al. 2018. Impact of genetically engineered maize on agronomic, environmental and toxicological traits: a meta-analysis of 21 years of field data. Scientific Reports 8
  • Rizzi, A. et al. 2012. The stability and degradation of dietary DNA in the gastrointestinal tract of mammals: implications for horizontal gene transfer and the biosafety of GMOs. Critical Reviews in Food Science and Nutrition 52:142–161.
  • Scheideler et al. 2008. Evaluation of nutritional equivalency of maize grain from DAS-Ø15Ø7-1 (Herculex I) in the diets of laying hens. Journal of Applied Poultry Research 17:383–389.
  • Shelton et al. 2002. Economic, ecological, food safety, and social consequences of the deployment of Bt transgenic plants. Annual Review of Entomology 47:845–881
  • Schrøder et al. 2007. A 90-day safety study of genetically modified rice expressing Cry1Ab protein (Bacillus thuringiensis toxin) in Wistar rats. Food and Chemical Toxicology 45:339–349.
  • Shimada et al. 2006. Bacillus thuringiensis insecticidal Cry1ab toxin does not affect the membrane integrity of the mammalian intestinal epithelial cells: An in vitro study. In Vitro Cellular and Developmental Biology 42:45–49
  • WHO/IPCS. 1999 Environmental health criteria: microbial pest control agent Bacillus thuringiensis. International Programme on Chemical Safety. Geneva, Switzerland.
  • Wiedemann et al. 2007. Effect of feeding cows genetically modified maize on the bacterial community in the bovine rumen. Applied Environmental Microbiology. 73:8012-8017
  • Wolfenbarger et al. 2008. Bt Crop effects on functional guilds of non-target arthropods: a meta-analysis. PLoS ONE 3:e2118
  • Yuan et al. 2013. Effects of genetically modified T2A-1 rice on the GI health of rats after 90-day supplement. Scientific Reports 3:1962
Posted in GMO | Tagged , , | 8 Comments

The concept of a “chemical-free lifestyle” is absurd

From Tim Minchin’s introduction to his brilliant song, “The Fence.”

Chemophobia is alive and well. It is difficult to get on the internet without celebrities, friends, and family members bombarding you with concerns about chemicals in your food, hygiene products, vaccines, etc. Indeed, being anti-chemical seems to be extremely fashionable at the moment, and you will often hear people talk about living a “chemical-free lifestyle” or trying to “avoid chemicals.” The problem is, of course, that everything is made of chemicals. Literally all matter is made of chemicals, and if you truly lived a chemical-free lifestyle, you would only live for a matter of minutes, after which you would die from a lack of oxygen.

I’ve written about this topic before, and I don’t want to spend this entire post belaboring the point, because it is a fundamental and basic fact of science that I shouldn’t have to explain. You are a biochemical machine that ingests and inhales chemicals (food, water, and air) and uses chemical processes to release energy from those chemicals so that the energy can be used to power your body. Everything you do is the result of chemical reactions. I could spend a long time explaining all of that in more detail, but for this post I want to focus on why it is important to understand that everything is made of chemicals and why we shouldn’t let people like the Food Babe get away with making claims like, “There is just no acceptable level of any chemical to ingest, ever.”

You see, when I point out the ridiculousness of trying to avoid chemicals, many people accuse me of pedantry and say that people who say that they are trying to avoid chemicals do know that everything is made of chemicals, they are just using the word chemical to mean “toxic” chemicals, or sometimes, “unnatural” or “synthetic” chemicals. Beyond the fact that they are distorting the definition of “chemical” to suit their own fears and biases, that response is very problematic, and I want to talk about why.

First, I’m not convinced that everyone is actually aware that everything is made of chemicals. Remember, the Food Babe was also concerned that airplane cabins didn’t have 100% oxygen, and she claimed that saying the words “Hitler” or “Satan” to water would change the water’s physical structure. So, at times we are clearly dealing with an extremely low level of scientific literacy and understanding.

Having said that, I can accept that most people probably do know that everything is made of chemicals, which brings me to my second and most important point. Using the word, “chemical” as shorthand for a “toxic” or “unnatural” or “dangerous” chemical creates a false dichotomy and fundamentally misrepresents chemistry. It ignores basic facts about chemistry, and in so doing, it dangerously misleads the public.

Let’s start with this notion of toxicity and assume that when someone says something like, “There is just no acceptable level of any chemical to ingest, ever” they really mean, “There is just no acceptable level of any toxic chemical to ingest, ever.” That statement is still fundamentally wrong, because the most basic concept of toxicology is that the dose makes the poison. There is no such thing as a “toxic” chemical; there are only toxic doses. Every chemical is toxic at a high enough dose and safe at a low enough dose. You can literally overdose on water if you drink too much of it in a short period of time. It is actually toxic to you at a high enough dose. Inversely, a few molecules of a chemical like cyanide won’t hurt you. Apple seeds contain cyanide, yet no one worries about accidentally ingesting one because the dose present in the seeds is far too small to be harmful to you. It’s not toxic at that dose. In other words, cyanide itself is not toxic to you as an organism. Rather, it becomes toxic at a high enough dose, just as water does.

As you can hopefully now see, even the concept of having a “[toxic] chemical-free lifestyle” is absurd, because all chemicals are toxic at a high enough dose. This concept proposes a simplistic false dichotomy between toxic chemicals and safe chemicals, while totally ignoring the fact that the dose is what makes something toxic. To be clear here, if you want to check the doses of chemicals present in your food, shampoo, etc., and also check the dose at which they become toxic to you, I have absolutely no problems with that, but that’s not what most people do. Rather, they view chemical toxicity as an entirely binary state. They view each chemical as either being toxic at any dose or safe at any dose, and they judge the safety of products merely by the presence or absence of a given chemical, rather than by looking at the dose. This simplistic view of toxicity is childish and dangerous.

Moving on, others use phrases like “chemical-free lifestyle” to mean a lifestyle that is free of “synthetic” or “unnatural” chemicals. This meaning is, however, even worse than the previous one. First, it once again assumes that chemicals can be placed into binary categories of “safe” or “not safe” without considering the dose. This is wrong. Both synthetic chemicals and natural chemicals have dose response curves. They are all toxic at high enough doses and safe at low enough doses.

This brings me to the second problem, namely, this argument is an appeal to nature fallacy. Nature is brutal and doesn’t give a crap about you. Nature will kill you in a million unpleasant ways, and the fact that something is “natural” tells you absolutely nothing about whether it is safe or beneficial. Remember earlier when we talked about cyanide? That’s a natural chemical. So is lead, aluminium, mercury, arsenic, formaldehyde, etc. Indeed, even if you lived in a pre-industrial society, you would naturally be exposed to most of these chemicals, and that would usually be fine, because they are all safe at low enough doses. The same thing is true when we talk about “synthetic” chemicals that scientists developed in laboratories. They are not inherently any more dangerous than a natural chemical. All chemicals are just combinations of atoms, and some of those combinations are only safe at very low doses while others are only dangerous at a very high doses, but all of them have a safe dose and a toxic dose. Where they originated is completely irrelevant.

As you can hopefully now see, statements about “chemical-free lifestyles” or “avoiding chemicals” aren’t wrong simply because everything is made of chemicals, but also because they represent a fundamental misunderstanding of chemistry and toxicology. These statements implicitly assume that some chemicals are always bad while others are always good, and that simply isn’t how chemistry works. The dose is what determines whether or not something is safe and chemophobia is irrational and misinformed. If someone tries to scare you about a chemical, ask them for the dose at which it is present in the item in question and the dose at which it becomes toxic. If they cannot answer both of those questions, then they either don’t know what they are talking about, or they are intentionally trying to mislead you.

Note for clarity: When I say that all matter is made of chemicals, I mean all matter at the atomic level and higher (obviously atoms themselves are made of subatomic particles).

Posted in Uncategorized | Tagged , | 13 Comments

Acupuncture is just a placebo

acupuncture does not workAcupuncture is an extremely popular form of complementary alternative medicine (CAM) that has even worked its way into many “integrative” hospitals. It is also fiercely defended by its believers. Unfortunately, it is not well defended by actual evidence, so I want to talk about that evidence and explain why acupuncture is a placebo. As usual, my point here is not simply to talk about acupuncture, but also provide a lesson in how to critically read the scientific literature. Acupuncture has been studied literally thousands of times, and, as a result, the literature is a mess and it is very easy to cherry pick studies to fit whatever view you hold. Therefore, you need to critically assess the literature and apply appropriate logical and scientific tools to arrive at a good conclusion.

TL;DR

Acupuncture is based on pre-scientific mysticism. It is supposed to work via the manipulation of acupoints, but scientists have been unable to find evidence that acupoints actually exist (i.e., they are not physiologically distinguishable from other points on the body). Additionally, there is no known mechanism through which acupuncture could work. Nevertheless, thousands of tests have been conducted. Meta-analyses and systematic reviews of these tests are extremely inconsistent, with little agreement among them. Many analyses failed to find evidence that it is better than a placebo, while others found a significant difference. However, the positive results usually had very small effect sizes, suggesting that the results were not clinically significant and were likely statistical flukes. Further, these studies also documented a large placebo component to the treatments. Additionally, several studies have documented a positive bias in the literature, with higher quality studies tending to produce more negative results. This lack of mechanism, large number of negative results (especially from high quality studies), inconsistency among studies, and small effect sizes all indicate that acupuncture is nothing more than a placebo.

Purported benefits

What does acupuncture actually treat? According to its disciples, pretty much everything. According to the Center for Integrative Medicine it treats allergies, depression, dysentery, numerous forms of pain, stroke, nausea, morning sickness, headaches, labor difficulties, and multiple other conditions. They also say that it can probably treat acne, alcoholism, palsy, asthma, diabetes, infertility, herpes, schizophrenia, whooping cough, and a dizzying array of other ailments. Erectile dysfunction is, of course, also on the list, because no miracle cure would be complete without it (I wonder where the needles go for that one).

Whenever you see a list like this, huge red flags should go up. This type of list is one of the hallmarks of quack remedies (details here). It is simply not possible that a single treatment is going to cure everything from infectious disease to recovery from stroke. Now, you might try to get out of this by saying, “well, just because it doesn’t work on all of these doesn’t mean that it doesn’t work on any of them.” Technically, that is true, but how are you supposed to know which ones it works on, and why should you listen to the people promoting it when they are clearly trying to deceive you about at least some of its benefits? When someone says something like, “putting needles in your skin will cure your herpes” they have just lost all credibility, and you should not be getting medical advice or treatments from them. In other words, at the very least, these types of lists should make you very skeptical.

It is also worth mentioning that one of the problems with alternative medicine (including acupuncture) is that it tends to be poorly regulated, and practitioners get away with making outlandish claims that lack evidence to support them (something actual doctors can’t and don’t do; Ryan 2017).

Implausible mechanisms

Before we get to the literature itself, we need to lay some groundwork. Acupuncture is based on the pre-scientific notion that there is a life force (or energy) know as qi, and the correct flow and balance of this life force keeps you healthy. Acupuncture then “works” by inserting needles into “acupoints” along “meridians” of the body to cure diseases by correctly directing the flow and balance of qi. In other words, it’s magic. Any treatment that is based on the notion that diseases are caused by energy imbalances, blocked energy, etc. is pseudoscience, and should be rejected. You don’t have a “life force” and energy imbalances and blockages don’t make you sick. That’s pre-scientific malarkey.

Now you may be tempted to suggest that the people who developed the method “thousands” of years ago didn’t understand the mechanism and explained it with their pre-scientific superstitions, but the method does work, they were just wrong about the cause. That’s technically possible, but we’d need some pretty good evidence to conclude that it was true, and that’s where we quickly start running into problems. You see, meridians and acupuncture points aren’t real things (Ramey 2001). Their number and position changes based on who you talk to, and they don’t map to any reliable underlying physiological structure. Also, it is worth mentioning that acupuncture as we know it is probably not nearly as old as most people think.

Nevertheless, you can find many papers whose titles and abstracts seem to disagree with what I just said, but when you actually start looking into them, it quickly becomes clear that acupoints don’t exist. This is one of the fascinating things about the acupuncture literature. People seem to desperately want it work, and the result is that there are hundreds of studies that spin fanciful tales without having that data to back them up.

Let me give you a few examples. Consider the paper “What is the Acupoint? A preliminary review of acupoints” by Li et al. (2015). This paper acknowledges very early on that acupuncture points aren’t supported by evidence and aren’t distinguishable from other parts of the body.

“At present, there is no persuasive evidence for the existence of acupoints. For example, their location or number and the evidence from histological studies for acupoints are unconvincing.”

It sounds like we should be done at the point, right? But the authors continue, “This review focuses on the function of acupoints from different perspectives, which might explain what an acupoint.” [sic] In other words, “there is no evidence that these things are real, but we want them to be real, so we are going to go ahead and write an entire paper about them as if they are real.”

That’s not how science works, but there are tons of papers like that. Zhou and Benharash (2014) is another good example of this. Their paper was published in the Journal of Acupuncture and Meridian Studies, which, as you can imagine, is pretty heavily biased towards acupuncture (it’s a quack journal). Nevertheless, they stated, “These observations confirmed that there were no particular structures that were unique to acupoints.” This fact is reiterated numerous times in the paper. Yet despite this fact, they latch onto the observation that there are usually nerves near acupoints and spend the whole paper talking about hypothetical mechanisms as if they are established facts. It is true that you can usually find nerves near acupuncture points, but there are nerves just about everywhere in the body, so it’s not particularly interesting. If acupoints were real things that had medical relevance, then they should be distinct and physiologically identifiable, but they simply aren’t, and that’s a huge problem.

This brings me to my next major point. Despite thousands of studies being conducted on acupuncture, no one has been able to demonstrate a mechanism through which it works. Oh, there are tons of hypotheses, but no one has actually been able to convincingly demonstrate a mechanism, and that’s another problem. It’s a standard that we wouldn’t accept for pretty much any other form of treatment. Imagine that your doctor described a drug, and when you asked what the drug actually does, they said, “No one knows. Scientists have looked at it for years and can’t figure it out, but trust me, it totally works.” You’d probably be pretty skeptical about that drug.

Now, to be clear, having an established mechanism is not 100% necessary to demonstrate that something works. You could still do it with really convincing clinical trials, but, as I’ll explain in the next section, the level of evidence required is much higher.

Low prior probability

Prior probability is a very important concept in science that I have previously talked about at length. Briefly, it is the probability that a given result could be true given everything else that we know about the system in question. In other words, we already know a lot about the human body, chemistry, etc. As a result, before a given treatment is tested, we can have a pretty good idea of how likely it is that the treatment could actually work, and the more unlikely it is, the higher the evidence bar is going to be. This is very much in line with the saying that extraordinary claims require extraordinary evidence, and it is important because it is very easy to get spurious results from scientific tests. Therefore, you need to judge how confident you should be in those results. If a conclusion is implausible based on everything else we know, then we need really robust studies, large sample sizes, and large effect sizes before we can conclude that the result is real.

Now, let’s apply that to acupuncture. Here is the situation: it is based on pre-scientific superstition rather than medical knowledge, the acupoints that are fundamental to how it is supposed to work don’t actually exist, and there is no known mechanism through which it works. Indeed, if you just stop and think about it for a second, it is pretty implausible. How likely is it really that poking needles into the skin can relieve pain, cure infectious diseases, help with childbirth, treat gastrointestinal problems, etc.? It doesn’t make sense based on everything else that we know. Therefore, the prior probability is very, very low, which means that we need some extraordinary evidence to match these extraordinary claims.

As I said earlier, you could always acknowledge that most of these treatments are implausible (or even impossible), but still argue that some of them have a higher probability, and I will grant you that some are more plausible than others, particularly pain. It is conceivable to me that putting needles in the skin could have some form of neurological effect that might temporarily reduce pain, but it’s still not likely, and I still want some very strong evidence (especially given a lack of known mechanism).

What would it take to convince me?

I’m finally almost ready to start looking at the literature, but before I do that, I want to lay out exactly what it would take to convince me that acupuncture is actually an effective treatment. I find this to be a very helpful exercise that I encourage you all to undertake regularly.

First, I would need a very consistent body of evidence showing that it is better than a placebo. To be clear, when I say, “consistent” I don’t mean that every single study will agree. There will always be statistical noise and bad studies, but if it actually works, then it should be obvious when you look at the literature. There should be very wide-spread, obvious, and undeniable agreement among studies. Also, these studies need to be large and well controlled. Finally, it needs to have a large enough effect size that it is clear that it is a real effect, not a statistical fluke. In other words, it should be substantially better than a placebo (i.e., there should be an obvious clinical benefit). These criteria are very reasonable and appropriate, especially given the lack of mechanism and low prior probability.

The literature is a mess

The scientific literature testing acupuncture is a mess. There are always disagreeing studies in any field, and you can always find at least a few papers that argue for pretty much any position, but I have rarely seen such an incomprehensible mess. There are thousands of studies, a huge portion of which are terribly designed. Tons of them lack adequate controls, most of them are tiny (though there are exceptions), designs are extremely inconsistent with numerous methods being used and outcomes being measured, and biases and conflicts abound. Indeed, although there is a strong bias towards publishing positive results in general, it seems particularly dominant in acupuncture studies. As I said earlier, there are entire journals devoted to it. Plus, there are several acupuncture institutes that publish regularly, and it is well established studies from China (which accounts for much of the literature) are heavily biased and often involve inappropriate methods, inaccurate reporting, and biased reviews (Vickers et al. 1998; Wu et al. 2009; Ma et al. 2012; Wang et al. 2014). Indeed, studies from China (and other Asian countries) almost always report positive results, which is in stark contrast to studies from other countries. To put that another way, even for the conditions that have a prior probability of virtually zero (e.g., infectious diseases), China (and other Asian countries) are cranking out positive results that research groups in other countries can’t replicate.

This is all very disturbing, because it means that there are tons of bad studies out there, and the literature is very biased. Indeed, if you read the work of John P. A. Ioannidis, who has spent much of his career studying biases and problems in the scientific literature, you will find that the acupuncture literature matches pretty much every quality that he says to be cautious of. Here is a quote from the abstract of his famous paper, “Why most published research findings are false” (which I discussed here).

“In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance.”

Nearly all of those conditions are met by the acupuncture literature. Indeed, an admittedly older review of the literature found that three fourths of acupuncture studies were of low quality, and low quality was associated with positive results (Ezzo et al. 2000). I have been unable to find a more recent study that actually estimates the percent of studies that are of low quality, but Deng et al. (2015) discussed many biases and methodological problems that are prevalent in the acupuncture literature, Linde et al. (2010) found that larger more robust studies tended to have fewer positive results than smaller studies (thus suggesting that much of the acupuncture literature is false positives from small studies), and the massive Ernst et al. (2011) review of reviews reported many low-quality studies.

Further, there is one really important source of bias that almost all acupuncture studies have. Namely, they aren’t double-blind. The person administering the acupuncture usually knows if they are giving real acupuncture or a placebo (e.g., sham needles or toothpicks). This could easily bias the results in a positive direction. Indeed, as I’ll talk about in a minute, it is well established that there is a huge placebo component to acupuncture, so it is entirely possible that slight differences in the behavior of the person administering the treatment could bias the results.

Numerous studies show that acupuncture does not work

Despite all the problems with the literature, we can still attempt to weed out bad papers and look at the randomized controlled trials with the highest standards and most rigorous methods. Here again, however, there are thousands of randomized controlled trials. As a result, it is very easy to cherry-pick studies, and citing individual studies is pretty pointless. Therefore, I am going to focus on meta-analyses and systematic reviews and ask that you do likewise and refrain from flooding the comments with cherry-picked studies. I’ve explained the hierarchy of evidence in more detail previously, but in short, meta-analyses and systematic reviews are the highest forms of evidence because they either attempt to review all relevant papers on a topic (for systematic reviews) or combine the data sets of multiple papers and run new analyses on those combined data sets (for meta-analyses). This, in concept, allows them to see overarching trends rather than statistical noise.

So what do these studies find? First, I want to acknowledge that studies comparing acupuncture to no acupuncture do show a “benefit” of acupuncture. That is hardly surprising, however, because virtually any treatment is “better” than no treatment. That is how placebo effects work (Finniss et al. 2010), and it is why the medical and scientific community defines effectiveness as being better than a placebo. To put it simply, if you do a test of a sugar pill vs no sugar pill, you will find a “benefit” of the sugar pill. Does that mean that the sugar pill is actually biologically active and is doing something useful? No, it’s just a placebo.

Understanding placebos is important, because as I will demonstrate in a minute, acupuncture has a strong placebo component (indeed, that seems to be the entirety of its effects), but many of the studies on it did not use a placebo (which is usually something like a sham needle that doesn’t actually penetrate the skin or even a toothpick). Thus, many of the studies supporting acupuncture were not properly controlled and are, therefore, unreliable. In contrast, numerous studies that were controlled have found that sham (placebo) acupuncture is just as “effective” as regular acupuncture. You can literally poke someone with a toothpick and get the same response as actual acupuncture, which is pretty damning evidence against acupuncture (here is one such example just so you can see what a study like that looks like: Cherkin et al. 2009).

Now, on to the actual reviews and meta-analysis. I can show you numerous systematic reviews and meta-analyses that fail to find that acupuncture performs better than a placebo (i.e., acupuncture doesn’t work). For example: Linde et al. 2011 (migraine prophylaxis), Davis et al. 2008 (tension head-aches), Lee and Ernst 2005 (surgery-related pain), Lee et al. 2005 (cancer-associate pain), Mayhew and Ernst 2009 (fibromyalgia), Zhang et al. 2010 (depression), Kong et al. 2010 (recovery from stroke), Smith et al. 2013 (inducing labor), Lim et al. 2006 (irritable bowel syndrome), etc. I could keep going, but hopefully I have made my point (also, see this list of Cochrane Reviews which paints a very bleak picture regarding the usefulness of acupuncture).

Nevertheless, you will, admittedly, not find it difficult to find meta-analyses and reviews that argue that acupuncture is actually more than a placebo. So, which do we trust? There are several things to consider here. First, we need to keep the aforementioned biases in mind. Second, we need to look for consistency. You may remember that this was one of my criteria for being convinced that acupuncture really works. As you may have guessed, however, that consistency is nowhere to be found (Ernst 2006). This was one of the chief conclusions of a very large systematic review of systematic reviews regarding acupuncture and pain (Ernst et al. 2011). Take a look at the tables in that paper. The reviews are all over the map. Indeed, the only condition for which there was consistent positive evidence from multiple high-quality reviews was for neck pain.

This is not what we would expect if acupuncture actually works. If it actually works, studies should consistently find that it works, but that’s not what we see. This is, however, exactly what we would expect if it is nothing but a placebo. We would expect a situation like this where (by a combination of chance and biases) some conditions occasional achieve positive results, but there is no consistency. Indeed, the fact that neck pain was the only condition with consistent results is very damning. Really think about this. Does it actually make sense that acupuncture works for neck pain, but not other types of musculoskeletal pain? No. That strains credulity. An editorial in the journal Pain (Hall 2011) described this well when the author said,

“Ernst et al. point out that the positive studies conclude that acupuncture relieves pain in some conditions but not in other very similar conditions. What would you think if a new pain pill was shown to relieve musculoskeletal pain in the arms but not in the legs? The most parsimonious explanation is that the positive studies are false positives.”

The final thing that must be considered here is the importance of effect size. You may recall that I specified that acupuncture should have a large effect size, and that Ioannidis (2005) warned that studies with small effect sizes are often spurious false positives. Thus, we should be cautious about saying that something works if it only shows a very small benefit.

There are two key concepts here that need to be understood to really evaluate effect sizes. The first is that P values (which are used to establish statistical significance) are probabilities, and they are often abused. The P value is the probability of getting a result as large or larger than the one you observed if there isn’t actually a difference in your groups. In other words, it is the probability that a result like yours could arise by chance (this assumes no bias or flaws in your experimental design). In biology, we usually say that something is statistically significant if it has a P value less than 0.05. In other words, if there is less than a 5% chance that a result like yours could arise by chance. Having a clear cut off like that has value, but people often make the mistake of treating 0.05 as a magical number that divinely arbitrates truth. Thus, if something has a P value of 0.06, it gets dismissed as non-significant, and if it has a P value of 0.04, it is automatically treated as a real result. That approach is silly. You should not be much more confident in a 4% chance than a 6% chance. Therefore, rather than blindly following P values, you should also look at confidence intervals or some other measure of variation, the actual size of the effect you observed, the sample size, etc. You need all of these pieces of information to really understand the result.

The other important concept here is that statistical significance and clinical or biological significance are not the same thing. Any difference between two groups becomes statistically significant with a large enough sample size, but that may not have any actual clinical relevance. It may be a difference that is too small to have any practical value (I talked more about P values and statistical significance here and here).

When we apply these concepts to acupuncture studies, we find many very small effect sizes. In other words, even when meta-analyses found a significant difference between sham (placebo) acupuncture and real acupuncture, the “benefits” of real acupuncture were quite small, often to the point that they have no clinical significance. To their credit, some authors have done a good job of acknowledging this. For example, an often-cited review and meta-analysis of acupuncture for pain (Madsen et al. 2009) stated,

“A small analgesic effect of acupuncture was found, which seems to lack clinical relevance and cannot be clearly distinguished from bias. Whether needling at acupuncture points, or at any site, reduces pain independently of the psychological impact of the treatment ritual is unclear.”

A review and meta-analyses for fibromyalgia made a similar statement (Langhorst et al. 2010):

“A small analgesic effect of acupuncture was present, which, however, was not clearly distinguishable from bias. Thus, acupuncture cannot be recommended for the management of FMS”

Nevertheless, not all authors have been this honest about their results, and the acupuncture literature is full of studies with tiny effects but grand claims (again, people seem to really want acupuncture to work). I want to talk about one particular study which is emblematic of this problem: Vickers et al. (2012) “Acupuncture for Chronic Pain: Individual Patient Data Meta-analysis.” When it came out, this study was spread wide and far by the press and was touted as concrete evidence that acupuncture works. When you actually look at the study, however, the situation is quite different. Both Dr. Steven Novella at Science-Based Medicine and Orac at Science Blogs have gone over this paper in detail, so I will give the short version.

This meta-analysis showed two things. First, both actual acupuncture and sham (placebo) acupuncture were “better” than no acupuncture (again, consistent with a placebo effect). Second, there was a very slight, but statistically significant difference between sham acupuncture and actual acupuncture. Let me be clear about what I mean by “slight.” On a pain scale of 1–10, the “benefits” of acupuncture vs. sham acupuncture were 0.5. In other words, they were too small for people to actually notice. Do you honestly think you can distinguish between a pain of 5.5 and 6.0? I doubt it. Indeed, this has actually been studied, and a review of pain in arthritis studies (Stauffer et al. 2011) found that a minimum of 0.7 was required for patients to detect it (usually more). In other words, a difference of 0.5 is not detectable by patients and is not clinically significant. Further, that difference is so tiny that it is extremely like that it could have resulted from biases, such as the fact that the trails were not double-blinded (i.e., the people administrating the acupuncture knew if they were giving a placebo or real acupuncture). It’s also worth mentioning that the study was conducted by the “Acupuncture Trialist’s Collaboration” so the authors had a fair amount of bias going into this.

Indeed, when you actually look at Vickers et al. (2012), it undeniably shows that almost the entire effect of acupuncture is from a placebo effect. Think about it, both sham and actual acupuncture had a large effect, but the difference between those two was imperceptibly small. In other words, the perceived benefits were due almost (if not entirely) to a placebo effect. Dr. Edzar Ernst, who has published many papers on acupuncture, stated,

“In my view, this meta-analysis is the most compelling evidence yet to demonstrate the ineffectiveness of acupuncture for chronic pain.”

 In the interest of fairness, the authors of the meta-analysis responded (Vickers et al. 2013), but their response is less than satisfactory. First, they do what many pseudoscientists do when criticized and incorrectly accuse their opponents of ad hominem attacks. They do eventually try to address the substance of the criticisms, but their rebuttals are less than convincing. For example, to address the argument that the slight difference could easily have been from a lack of blinding, they cited another paper by their group supposedly showing that acupuncture is better than sham acupuncture even when double-blinding is used (Irnich et al. 2002). At the risk of going down a side-tangent, I want to talk about this study for a second, because it once again nicely illustrates the type of shoddy science that is often used to support acupuncture.

To compare actual acupuncture with sham acupuncture in a double-blind design, one group received real acupuncture, while the other received lazer acupuncture (that’s a thing), but unknown to the administrator, the lazer’s bulb had been replace with a regular bulb that just made a red dot of light. This is not a good design for multiple reasons. First, using lazer acupuncture vs real acupuncture does not adequately blind patients, because in one treatment they feel pressure from a needle, and in the other, they don’t. Further, the fake lazer emitted a noise, thus making the different treatments obvious to patients. This is an awful design. To make things even worse, the person operating the lazer was always different from the person administering the actual acupuncture. Thus, the treatments were completely confounded with the person administering them, making the results impossible to interpret. It is entirely possible that the ones giving actual acupuncture simply had better bed-side manners, and that resulted in the difference (indeed, that seems likely, since they had years of experience, whereas the guy with the lazer wasn’t even certified). The point that I am trying to make here is that this is the type of evidence that people use to defend acupuncture. This type of garbage is the best that they have, and the fact that they think it is good evidence clearly reveals their biases.

Now, maybe you haven’t been convinced by any of this. Maybe you really desperately want acupuncture to work, and therefore you reject my arguments that the disagreement in the literature is a problem. If that is the case, then the best you could possibly say, with a really generous interpretation of the literature, is that there is wide-spread disagreement among studies, there are only a handful of afflictions with reasonably consistent results, and even for those, the benefits are so small that most people won’t notice them. Indeed, even the studies that argued that real acupuncture is better than sham acupuncture also found that almost the entire difference between acupuncture and no acupuncture could be explained by a placebo effect.

That is simply not compelling evidence, especially given the lack of mechanism and lack of evidence for even the existence of acupoints!

Think about it this way. Imagine for a second that we are talking about a pharmaceutical instead of acupuncture. Would you really take a drug if there was no known mechanism through which it could work, the physiological apparatus that it was supposed to interact with didn’t exist, there were numerous studies showing that it was no more than a placebo, and even the studies that argued that it works found such a tiny effect that you probably wouldn’t notice it? Would you honestly think that evidence was compelling?

Side effects

I want to quickly point out that acupuncture is not without side effects (Ernst et al. 2011; Xu et al. 2013). To be clear, most side effects are minor, but serious ones do occur, including organ trauma and even death. Your odds of having a serious problem are admittedly quite low, but why take the risk for something that is just a placebo? All actions have risk, and you need to weigh the risks against the benefits. In this case, the risk is admittedly low, but the benefit is non-existent (it’s a placebo), so why take the risk?

Counter arguments

Before I conclude this post, I want to briefly address some of the more common responses to posts like this (please don’t waste my time in the comments with arguments I’ve already addressed).

Anecdotes

This is probably the most common response. People “know” that it works, because they tried it and felt better. Anecdotes are not, however, good evidence of causation. As I have explained at length, you probably felt better because of a placebo effect. Indeed, saying “I did X, then felt better, therefore X works” is a logical fallacy known as post hoc ergo propter hoc. It is not valid reasoning (details on why anecdotes aren’t good evidence here and here).

“It’s been used for thousands of years, so it must work”

This is known as an appeal to antiquity fallacy. The fact that something was used for a very long time does not mean it works. For example, tobacco was used medicinally for centuries before we found out that it is very harmful. Similarly, leeches, bloodletting, and countless other insane treatments were used for very long periods of time before being abandoned. I honestly don’t understand why people think this is a good argument. The fact that acupuncture predates science is an argument against it, not for it. Also, it is worth mentioning that China had actually largely abandoned acupuncture until gullible westerners took an interest in it.

“But some hospitals and doctors recommend it”

This is a form of the appeal to authority fallacy. For one thing, there are also many who agree that it is bunk. Additionally, in recent years there has been a disturbing infiltration of quack treatments into hospitals, medical schools, and medical organizations (largely driven by public demand for those treatments). This does not, however, validate those methods. For example, my university recently opened a healing touch clinic. Does that mean that there is actually good scientific evidence for magical healing touch therapies? No, it means my university figured out how to make more money from gullible people. You need actual evidence to show that something works, and as I have shown, that evidence does not exist for acupuncture (note a popular publication by WHO touting the benefits of acupuncture is often cited as evidence, but that publication was retracted in 2014 because it wasn’t based on evidence).

“A placebo effect is still an effect”

This argument asserts that even if acupuncture is just a placebo effect, it still helps people. It would take me an entire post to explain the problems with this in detail, but, here are some highlights. First, this argument is inane. Saying that something works as long as it produces a placebo effect makes no sense. It disregards fundamental concepts about how we conduct research and define effectiveness. Indeed, it is nothing more than a cop-out to dismiss a lack of evidence for a treatment that someone wants to believe in.

Second, this argument proposes that doctors should lie to their patients about the effectiveness of treatments that don’t actually work. That is a huge violation of ethical practices.

Finally, this argument misunderstands placebo effects, because they cover far more than simply thinking that you are going to get better, and it is not at all clear that placebos are worth much on their own. Dr. David Gorski at Science-Based Medicine explains all of this in more detail.

What’s the harm?

At this point in a post like this, many people fall back on simply asking, “what’s the harm? Does it really hurt anything if people want to believe in and use acupuncture?”

Yes, it does. For one thing, as stated previously, acupuncture does have adverse effects, including (rarely) death. Second, people may be inclined to use acupuncture instead of treatments that actually work. Third, I believe strongly in the benefits of knowledge, and continuing to act as if this pre-scientific hogwash is real and beneficial is antithetical to the goal of progressing our knowledge and understanding of the universe. This brings me to my final point: because the public wants acupuncture to be true and keeps spending money on acupuncture, scientists keep studying it. We have now wasted untold millions of dollars and decades of research on studying a treatment that doesn’t work. Imagine if all that time and money had been spent improving cancer treatments, studying neurological disorders, designing better anti-viral drugs, etc. There are so many better ways to spend that money, yet each year, millions more are wasted studying this placebo. That is a problem.

Conclusion

I will end with the quote from Friends of Science in Medicine’s review of acupuncture which summed things up better than I could,

“Acupuncture has been studied for decades and the evidence that it can provide clinical benefits continues to be weak and inconsistent. There is no longer any justification for more studies. There is already enough evidence to confidently conclude that acupuncture doesn’t work. It is merely a theatrical placebo based on pre-scientific myths. All health care providers who accept that they should base their treatments on scientific evidence whenever credible evidence is available, but who still include acupuncture as part of their health interventions, should seriously revise their practice. There is no place for acupuncture in Medicine.”

Related posts on evaluating the scientific literature 

 Suggested further reading 

Literature cited

  •  Cherkin et al. 2009. A randomized trial comparing acupuncture, simulated acupuncture, and usual care for chronic low back pain. Archives of Internal Medicine 169:858–866.
  • Davis et al. 2008. Acupuncture for tension-type headache: a meta-analysis of randomized, controlled trials. Pain 9:667–677.
  • Deng et al. 2015. Is acupuncture no more than a placebo? Extensive discussion required about possible bias. Experimental and Therapeutic Medicine 10:1247–1252.
  • Ernst 2006. Acupuncture — A critical analysis. Journal of Internal Medicine 259:125–137.
  • Ernst et al. 2011. Acupuncture: Does it alleviate pain and are there serious risks? A review of reviews. Pain 152:755–764.
  • Ezzo et al. 2000. Is acupuncture effective for the treatment of chronic pain? A systematic review. Pain 86:217–225.
  • Finniss et al. 2010. Placebo effects: Biological, clinical and ethical advances. Lancet 375:686–695.
  • Hall 2011. Acupuncture’s claims punctured: not proven effective for pain, not harmless. Pain 152:711–712.
  • Ioannidis 2005. Why most published research findings are false. PLoS Medicine 2:e124.
  • Irnich et al. 2002. Immediate effects of dry needling and acupuncture at distant points in chronic neck pain: results of a randomized, double-blind, sham-controlled crossover trial. Pain 99:83–89.
  • Kong et al. 2010. Acupuncture for functional recovery after stroke: a systematic review of sham-controlled randomized clinical trials CMAJ 182:1723–1729.
  • Langhorst et al. 2010. Efficacy of acupuncture in fibromyalgia syndrome—a systematic review with a meta-analysis of controlled clinical trials. Rheumatology 2010;49:778–88.
  • Lee and Ernst 2005. Acupuncture analgesia during surgery: a systematic review.
  • Pain 114:511–517.
  • Lee et al. 2005. Acupuncture for the relief of cancer-related pain—a systematic review. European Journal of Pain 437–444.
  • Li et al. 2015. What is the acupoint? A preliminary review of acupoints. Pain Medicine 16: 1905–1915.
  • Lim et al. 2006. Acupuncture for treatment of irritable bowel syndrome. Cochrane Database of Systematic Reviews 18:CD005111.
  • Linde et al. 2011. Acupuncture for migraine prophylaxis. Cochrane Database of Systematic Reviews (1):CD001218.
  • Linde et al. 2010. How large are the nonspecific effects of acupuncture? A meta-analysis of randomized controlled trials. BMC Medicine 8:75.
  • Ma et al. 2012. Epidemiology, quality, and reporting characteristics of systematic reviews of acupuncture interventions published in Chinese journals. Journal of Alternative and Complementary Medicine 18.
  • Mayhew and Ernst 2009. Acupuncture for fibromyalgia—a systematic review of randomized clinical trials. Rheumatology 46:801–804.
  • Ramey 2001. Acupuncture points and meridians do not exist. The Scientific Review of Alternative Medicine 5.
  • Ryan 2017. Acupuncture, ACC and the Medicines Act. New Zealand Medical Journal 130.
  • Smith et al. 2013. Acupuncture for induction of labour. Cochrane Database of Systematic Reviews 15:CD002962.
  • Stauffer et al. 2011. Definition of nonresponse to analgesic treatment of arthritic pain: An analytical literature review of the smallest detectable difference, the minimal detectable change, and the minimal clinically important difference on the pain visual analog scale. International Journal of Inflammation 2011.
  • Vickers et al. 1998. Do certain countries produce only positive results? A systematic review of controlled trials Controlled Clinical Trials 19:159–166.
  • Vickers et al. 2012. Acupuncture for Chronic Pain Individual Patient Data Meta-analysis. JAMA 172:1444–1453.
  • Vickers et al. 2013. Responses to the Acupuncture Trialists’ Collaboration individual patient data meta-analysis. Acupuncture in Medicine 31:98–100.
  • Wang et al. 2014. Positive results in randomized controlled trials on acupuncture published in Chinese journals: A systematic literature review. Journal of Alternative and Complementary Medicine 20.
  • Wu et al. 2009. Randomized trials published in some Chinese journals: how many are randomized? Trials 10:46.
  • Xu et al. 2013. Adverse events of acupuncture: A systematic review of case reports. Evidence-based Complimentary Medicine 2013:581203.
  • Zhang et al. 2010. The effectiveness and safety of acupuncture therapy in depressive disorders: Systematic review and meta-analysis. 124:9–21.
  • Zhou and Benharash 2014. Effects and mechanisms of acupuncture based on the principle of meridians. Journal of Acupuncture and Meridian Studies 7:190-193.
Posted in Vaccines/Alternative Medicine | Tagged , , , , , | 18 Comments