The Logic of Science

Why are there so many reports of autism following vaccination? A mathematical assessment

Advertisements

The idea that vaccines cause autism is one of the most persistent myths that I have ever encountered, and it seems that no amount of evidence will ever cause it to disappear. Indeed, I recently wrote a lengthy post in which I thoroughly reviewed the scientific literature on this topic, and I showed that there are no high quality studies supporting this myth, but there are multiple very large studies that debunked it. Nevertheless, many people responded to the post by insisting that vaccines must cause autism because there are so many cases of parents reporting the onset of autism shortly after vaccinating. They were adamant that these anecdotes could not be chance results and must mean that vaccines cause autism. To quote one commenter,

“How can you sit there and say that observing an adverse reaction AFTER a vaccine is given doesn’t mean it was the vaccine? So, what…? It’s just a random coincidence? I think not.”

Given how common this argument is, I want to look at these anecdotes and what they actually mean, and, as part of that, I want to actually do some math and calculate the probability of a child developing autism shortly after being vaccinated if the vaccine isn’t responsible. In other words, I want to calculate the odds of observing autism “AFTER a vaccine is given” just by “random coincidence.”

The problem with anecdotes
Anecdotes are extremely problematic for multiple reasons. I’ve explained them in detail before, so I will just discuss the core problem here. Namely, using anecdotes as evidence of causation commits a logical fallacy known as post hoc ergo propter hoc. This fallacy occurs whenever you say “X happened before Y, therefor X caused Y” (e.g., “Billy was vaccinated shortly before showing the first signs of autism, therefore the vaccine caused the autism”). The problem is that just because one thing happens before something else does not mean that the two are related. It could, in fact, be a complete coincidence (more on that later). Many people often become very indignant when I say that (as the commenter that I quoted did), but these aren’t arbitrary rules that I have made up. This is just how logic works. The fact that X happened before Y does not in any way demonstrate that X caused Y. You need additional information (such as knowledge of confounding factors) before you can assume that they are related.

Here is an example that I often like to use. Imagine that I fill my car with fuel and it breaks down a mile later. Can I assume that the fuel was bad and that’s what made my car break down? Obviously not. There are lots of parts on a car that can break, and it is entirely possible that one of them just happened to break shortly after fueling up. In other words, it could my be a complete and total coincidence. Think about it, thousands of people fill up their cars every day, and thousands of people break down every day. So it is inevitable that there will be lots of cases where someone just happens to break down right after filling up. The fact that fueling up happened before breaking down does not mean that fueling up caused me to break down, and that exact same logic applies with vaccines and autism.

To give another example, imagine that someone takes a new medication, then has a heart attack the next day. Can we conclude that the medicine caused the heart attack? No, we can’t. Heart attacks are extremely common, so it shouldn’t be surprising that, just by chance, some people will have heart attacks shortly after taking a medicine, even if the medicine didn’t cause the heart attack.

How to study anecdotes
Although anecdotes are not very useful as evidence of causation, they can be very useful as starting points for scientific investigation. To go back to my heart attack example, if there were multiple reports of people having heart attacks shortly after taking the medication, doctors and scientists would take those reports seriously, and they would use them as the justification for doing a study. This is a crucially important point: the only way to actually know whether or not the medication causes heart attacks is to compare heart attack rates in people who do and do not take the medicine (or at least compare different doses) while controlling confounding factors. Only then, once you have controlled the confounding factors, can you conclude that the relationship is causal rather than simply being a coincidence. In other words, you have to control all other possibilities and compare two treatment groups before you can conclude that the medicine is the cause. Otherwise, you can never be sure that it’s not just a coincidence.

Now, if we go back to vaccines and autism, we find an identical situation. The fact that many parents have observed autism shortly after a vaccine does not in itself indicate that vaccines cause autism, but it does give scientists something to investigate. In other words, to test those anecdotes and find out whether they are arising from chance or from a causal relationship, we need to compare autism rates among children who did and did not receive a given vaccine. Scientists have, in fact, done that numerous times, which is why anti-vaccers’ comments are a bit strange. I constantly encounter anti-vaccers who insist that scientists aren’t taking parents seriously and haven’t examined the reason that autism often seems to follow vaccines. For example, someone made the following comment on my post about the scientific studies on vaccines and autism.

“In other words, there is a high enough number of parents reporting an immediate and dramatic and permanent change in the behavior of a child, that the link was established. This is the core of what needs to be understood and studied. It also appears that it continues to be ignored. In other words, until we study why these reports exist, and in the numbers that they exist, and focusing on just those particular children, we have not demonstrated the science needed to make a conclusion.”

Notice how the commenter suggests that scientists have ignored parents reports and aren’t studying those reports. That is a rather bizarre accusation given that we have multiple very large studies on whether or not vaccines cause autism. In other words, we know that the anecdotes are wrong because scientists took parents’ concerns seriously, carefully tested vaccines, and repeatedly found that there is no relationship between vaccines and autism. Further, just to be clear, scientists didn’t just do one small study then call it quits. Rather, they have looked it this from multiple angles such as testing different vaccines, examining the effects of age at vaccination, studying children that are at a high risk of autism, testing different doses of vaccines, etc., and some of those tests have been enormous (the largest used over 1.2 million children!). So scientists absolutely took parents’ concerns seriously, but it turned out that the parents’ concerns were unnecessary. In other words, the issue isn’t that scientists are ignoring parents. Rather, the issue is that they didn’t find the results that anti-vaccers wanted them to find. Scientists have very carefully examined vaccines to see if the anecdotes actually represent a causal relationship, and anti-vaccers are simply refusing to accept their results.

See the text for sources and more details.

Looking at the math
Despite the fact that anecdotes can’t demonstrate causation and the fact that multiple studies have shown that the anecdotes are in error, many people continue to insist that the temporal relationship between vaccines and autism simply can’t be a result of chance. So let’s look at that with some simple math. What I want to do is calculate how many times we expect the first signs of autism to closely follow vaccines if vaccines don’t actually cause autism.

Note: I will explain the math below, but for your convenience, I have laid it all out in the image above.

According to the CDC, there are roughly 3,988,000 children born in the US each year  (Hamilton et al. 2015). This gives us a nice annual cohort that I want to follow. Over 90% of children in the US receive their full vaccine schedule (Elam-Evans et al. 2014), so that gives us 3,589,200 vaccinated children per annual cohort (0.9*3,988,000). Now, in the US, 1 in 68 children develop autism (Christensen et al. 2016), so if vaccines and autism are not in any way related, we expect that 1 in 68 of our 3,589,200 children will develop autism. Thus, we should have 52,782 children each year who have autism and are fully vaccinated (3,589,200/68).

In roughly 80% of cases, parents first notice the signs of autism prior to a child’s second birthday, and usually not until after around 6 months old (Giacomo and Fombonne 1998). Therefore, for our 52,782 children, there should be 42,226 for which parents first noticed the signs of autism between 6 and 24 months old (0.8*52,782). Now, let’s assume for a minute that there is an equal probability of the first signs of autism appearing on any day during that period (I’ll talk about that assumption in a minute). As a result, we expect 77 children to show the first signs of autism on any given day (42,226 children/548 days).

Now, let’s bring all the pieces together. According to the recommended CDC vaccination schedule, most children receive vaccines on at least 2 days between age 0.5 and 2 years. So if we use that as a conservative estimate, then each year, we expect 154 children (77 cases*2 days) in the US to show the first signs of autism within 24 hours of being vaccinated just by chance (that number goes up dramatically if you spread the vaccines out over more than 2 days). Similarly, each year 1,079 children should show the first signs within 1 week of receiving a vaccine, and 4,623 should show the first signs within 1 month of receiving a vaccine. In other words, so many children develop autism and so many children are vaccinated, that even though vaccines do not cause autism, we still expect there to be hundreds or even thousands of cases where the signs of autism were first noticed shortly after receiving a vaccine (note again that this is the same situation as my fuel example earlier). So we do, in fact, expect there to be lots of coincidences where the detection of autism just happened to follow vaccination.

To be fair, my calculations obviously include two major assumptions, so let’s talk about those for a second. First, I assumed that vaccination only took place on two days, but that is actually a conservative estimate. If everyone vaccinated over 3 days, for example, then we expect there to be 231 annual cases of autism appearing within 24 hours of a vaccine.

Second, I assumed that the rates of autism detection were constant over the period we were talking about. That assumption is clearly false. In actuality, the detection probability goes up over time, but given that one of the days of vaccination is usually early in our time frame and the other is late, that should largely balance out. Also, realize that right now we are quibbling over the exact numbers that I calculated, rather than my central result. In other words, the exact numbers are almost certainly off, but the central point stands (i.e., we expect there to be lots of cases where, just by chance, autism is detected shortly after a vaccine).

Additionally, we would actually expect the odds of a parent noticing the symptoms of autism to skyrocket shortly after a vaccine is administered. Many parents are very concerned about a vaccine harming their child, and, as a result, they will tend to watch their children very closely after vaccinating them (even if they don’t consciously realize that they are doing so). Thus, they are far more likely to notice an early sign of autism that they might have missed if they hadn’t been watching their children so closely. To give an analogy, after people buy a new car, they often start seeing that model and paint job everywhere, but that model isn’t actually any more abundant than it was before, it’s just that their brains notice it because they are thinking about it (consciously or subconsciously). Even so, you are far more likely to notice an early sign of autism if you are worried about it. So in actuality, my numbers are likely underestimates rather than overestimates.

Finally, you may be thinking, “but those numbers are lower than actual number of cases of autism that follow vaccination each year,” to which I have several replies. First, again realize that these numbers are not precise and the true values are likely much higher. Second, where are your sources that the rates are much higher? I’m betting that your sources are simply collections of anecdotes, in which case, you don’t actually have any idea how often autism is detected within a few days of being vaccinated. It may seem much higher than it really is simply because you’re getting your information from internet echo chambers. This brings me to my final and most important point. I completely agree that this argument does not prove that vaccines don’t cause autism. Rather, it simply shows that we expect there to be lots of cases where the detection of autism follows vaccination just be chance. The only way to actually know whether the true rates are higher than the rates expected just by chance is (you guessed it) to do a large study, which, once again, is exactly what scientists have done multiple times.

Note: please read this post before arguing that we need a fully vaccinated vs. full unvaccinated study.

But aren’t autism rates increasing?
As I wrote this post, I could already hear peoples’ keyboards furiously clicking away and arguing that I must be wrong because autism rates have increased over time. So let’s talk about that for a minute. First, as I explained here, at least a large portion of the increase has been due to diagnostic changes, rather than an actual increase (i.e., people who would not have been considered autistic 20 years ago are considered autistic today;  Rutter 2005; Taylor 2006; Bishop et al. 2008; Baxter et al. 2015; Hansen et al. 2015). Second, that doesn’t change the math nor does it refute the numerous large studies that failed to find any evidence of vaccines causing autism. Remember, correlation is not the same as causation. Even if actual autism rates truly are going up, that wouldn’t mean that vaccines are the cause, and in fact, we know that vaccines aren’t the cause because we have so thoroughly studied this.

Summary
In conclusion, anecdotes cannot demonstrate causation, but they can be useful as starting points for further research. In the case of vaccines and autism, that research has been thoroughly conducted, and it has overwhelmingly shown that vaccines do not cause autism. Therefore, the reports of autism being detected shortly after vaccination must be arising by chance. Although many people protest that notion, it is not at all surprising given the number of children who receive vaccines and the number who develop autism. Further, I have mathematically demonstrated that even if vaccines do not cause autism, we expect there to be a large number of cases where, just by chance, autism is detected shortly after the administration of a vaccine.

Related Posts

Literature Cited

Advertisements

Advertisements