Another terrible anti-vaccine study bites the dust

Lately, anti-vaccers have been touting a “new” vaccinated vs. unvaccinated study that purportedly shows that vaccines are associated with all manner of detrimental health conditions. I put the word “new” in quotes, because this study was actually accepted for publication once before, but the journal that had accepted it (Frontiers in Public Health) retracted it before it actually came out. Following that retraction, the authors managed to get it published in the Journal of Translation Science as, “Pilot comparative study on the health of vaccinated and unvaccinated 6- to 12-year-old U.S. children.” That journal has now also retracted it, but I somehow doubt that is going to matter to anti-vaccers. This is, after all, the group that continues to praise Andrew Wakefield, despite his work being retracted due to extremely clear evidence that he falsified data, violated ethical rules, and was in it for the money. Therefore, I want to actually go through this study and explain why it is absolutely terrible. As usual, my goal is provide a worked example of how to critically analyze scientific papers, rather than simply addressing this one study (see my previous posts here, here, here, and here).

Note: I started writing this post several days ago, then became busy with my actual research, and by the time I got back to it, several excellent blogs/sites had already dealt with it, but since I had already started it, I figured I might as well finish it and add my two cents to the discussion.

Biased sampling design

Before I talk about the methods used in the paper itself, I need to talk about some basic concepts in designing studies. The first rule of experimental design is that you need to get representative samples, because failing to get a proper representation of the population will give a biased result. Imagine, for example, that I conducted a poll to see how popular Donald Trump was, and to do that, I sent the poll to multiple Republican organizations and asked them to distribute it among their members. Obviously, I would get a very biased result, because my sampling was not representative of the entire population (i.e., I sampled a group that predominately likes Trump, rather than sampling the population as a whole). Conversely, if I had sent that poll to liberal groups, I would have gotten a result that was biased in the opposite direction. Do you see the point? If you use a biased sampling method, you will get a biased result. That is why it is so important to randomize your sampling rather than doing what is known as “convenience sampling.” This is where you get your data from a group that is convenient, rather than a group that is representative. For example, if I wanted to conduct a study on how people felt when eating organic vs traditional food, it might be convenient for me to stand outside of a Whole Foods and poll people as they exit, but that would obviously be an extremely biased design. Of course, they are going to say that eating organic makes them feel healthier, that is why they are there. So whenever you are reading a paper, take a good look at how they did their sampling, and make sure that it wasn’t biased (note: when I said that this was rule #1, I was being literal, this is literally the first thing that I was taught in the first stats course I ever took).

So, how did this paper do its sampling? You probably guessed it; it used convenience sampling.

“the object of our pilot study was not to obtain a representative sample of homeschool children but a convenience sample of unvaccinated children of sufficient size to test for significant differences in outcomes between the groups.”

That’s not how this works. Statistical tests are not magic wands. They rely on strict assumptions, one of which is that your sampling was done correctly. If you put garbage in, you are going to get garbage out. So, saying that you used convenience sampling so that you could run statistics makes no sense, because the usage of convenience sampling prevents you from drawing valid statistical inferences.

To give you some more details about what this study did, it sent a survey to homeschool groups and asked them to distribute it among their members. This survey asked various questions about the children’s vaccination status, health, conditions during pregnancy, etc. This is obviously a huge problem, because, as the paper itself admits, there is a disproportionate number of anti-vaccers among homeschoolers. Indeed, it is not at all difficult to imagine anti-vaccers eagerly sending the survey to their anti-vax friends and telling them to fill it out (I have seen this behavior on similar online surveys). Thus, the paper clearly had a biased sample, so of course it found that vaccines cause problems. When you use an extremely biased sampling design like this, you expect to get a biased result, just as I would expect to find that Trump is popular if I used a biased sample of Republicans. Given this design, it would have been shocking if the paper found anything other than “vaccine injuries.”

We could actually stop right here and be done with this paper, because this biased sampling design completely precludes any meaningful comparisons. We can already see that the results are meaningless, but wait, there’s more.

All the results were self-reported

Not only did this study bias the people that it sampled, but it also used an extremely unreliable method of getting data from them. You see, the survey was entirely, 100% self-reported. Parents never had to send in any medical documents. Rather, they simply reported the vaccines their child received, the health conditions, etc. This is a terrible data collection design because memories are notoriously easy to influence, and self-reporting like this frequently fails to give reliable results (especially when using a biased sample).

The authors described several ways that they tried to eliminate these biases and control the reliability issues, but they were not at all adequate. First, they said that parents were instructed to consult their actual medical records, rather than relying on memories. There is, however, absolutely no guarantee that parents actually did that. People are lazy and generally bad at following instructions, so it is really hard to believe that all the parents in this study actually looked up their child’s medical records. Similarly, the authors told parents only to report conditions that were actually diagnosed by a physician, but once again, there is no way to know if parents actually did that, and, in fact, it is extremely likely that parents didn’t follow that. Imagine that you have an anti-vaccine parent who is convinced that vaccines are dangerous and injured their child, but since they don’t trust doctors, they never actually had the condition diagnosed. Do you honestly think that this parent isn’t going to jump at the chance to report their child’s “vaccine injury” even though they never got an official diagnosis? People lie on medical forms all the time, why should this be any different? Further, even if we assume that no parents deliberately lied, that doesn’t address the issue of “alternative” health practitioners. A parent may have had a condition “diagnosed” by a naturopath, which in their mind, counts as a medical diagnosis even though it’s actually nothing of the kind.

Now, you may say that this is all very speculative and I can’t actually prove that parents didn’t strictly obey the rules set out by the authors, and you are correct, but you’re missing the point. Reliability is one of the requirements for a good scientific study, and I shouldn’t have to assume that parents reported the data accurately. The fact that I am forced to make that assumption makes this study unreliable. In other words, I don’t know that parents relied on memories, lied, used the diagnoses of pseudo-physicians, etc., but I also don’t know that they didn’t do any of those things, and that is the problem. It means that we have no reason to be confident that these data are accurate. In contrast, if actual medical records had been supplied by the parents, then we would know that the reports are accurate. Additionally, it is worth mentioning that it would not take many dishonest and/or lazy parents to bias the study. Their sample sizes were quite small (especially for the children who had certain conditions), so just a handful of parents who provided bad info could seriously skew the results.

To be fair, the authors state that they chose not to collect actual records simply because after talking to several homeschool groups, the realized that doing so would result in a much smaller sample size and, therefore, prevent them from doing the study. It is probably true that they had no choice but to do it this way, but that doesn’t automatically make this method valid. In other words, given the choice of either not doing the study at all or doing the study by using an extremely unreliable method, they should have chosen the former. Unreliable data are often worse than not having any data at all.

Finally, the authors said, “Dates of vaccinations were not requested in order not to overburden respondents and to reduce the likelihood of inaccurate reporting,” but this seems totally backwards and nonsensical to me. Think about it. They already instructed the parents to look at the medical records rather than relying on memories, so if the parents already have the forms in hand, how is it an “overburden” to ask them to write down the date? Indeed, asking for a date seems like it should make parents more likely to actually consult their records, rather than going from memory. It’s almost like the authors tried to make this study as terrible as possible.

Differing numbers of doctor visits

There is one final flaw in their sampling/data collection that needs to be addressed. Namely, anti-vaccine parents and pro-vaccine parents generally differ in the frequency with which they take their children to the doctor. Indeed, this study even found that these groups differed in how frequently they visited the doctor. This is another big problem, because if one group visits the doctor more often, then you expect them to be diagnosed with more conditions by mere virtue of the fact that they see doctors more regularly (thus maximizing the odds of detecting any problems). This automatically biases the study towards higher numbers of problems in vaccinated children.

Statistical nightmares

Finally, we get to the statistics, and they are a disaster. The first thing to note is that they compared vaccinated and unvaccinated for over 40 conditions. That is potentially problematic, because with that many tests, you expect to get some false positives just by chance. This is what is known as a type I error. I have explained this in detail elsewhere (here, here, and here), so I’ll be brief. Statistical tests give you the probability of getting a difference as large or larger than the one that you observed if there is not actually a difference between the two groups from which your samples were taken. Typically, in biology, we set a threshold at 0.05 and say that if the probability (P value) is less than 0.05, then the result is statistically significant. In other words, for statistically significant results, there is less than a 5% chance that a result that is identical or greater could arise by chance. If you think about that for a second though, it should become obvious that you will sometimes get false positives. Further, the more tests that you run, the higher your chance of getting a false positive becomes. Therefore, whenever you are using lots of different tests to address the same question (e.g., “are vaccines dangerous?”) you should adjust your significance threshold (alpha) based on the number of tests you run. However, the authors of this study failed to do this, and given the enormous number of tests they ran, some of their results are probably just statistical flukes. (on a side note, their tables are also very deceptive, because they only showed the results that were significant or nearly significant, which makes it look like vaccines were significant for almost everything, when in reality, only a handful of their 40+ conditions were significant. This is not an honest way to present the results).

Additionally, beyond testing numerous different conditions, they also went on a statistical fishing trip with how they structured the tests. For many conditions, they made comparisons among fully vaccinated, partially vaccinated, and totally unvaccinated children, then also made comparisons among all children who received at least one vaccine, and all children who receive no vaccines. That is a problem because, again, the more tests you do, the more likely you are to get false positives. You can see why this is a problem when you look at things like ADHD and ASD. When they used all three groups, ADHD was not significant, and ASD was barely significant (though it wouldn’t have been if they had controlled their type I error rate correctly), but when you jump down to the tests with fully and partially vaccinated children lumped together, suddenly you get stronger significances (if we ignore the type I error problem). Thus, by doing multiple tests, they were able to get one to be significant. That is not ok. It is what is known as “P hacking.” You can’t just keep manipulating and retesting your data until something significant falls out. The correct way to do this would have been to define the groups ahead of time (a priori) then only run the comparisons on those pre-defined groups.

Before I move on, I also want to point out that that lumping the partially vaccinated and fully vaccinated children makes little sense, because the partially vaccinated group should be all over the place. For example, surely, a child who only received one vaccine would be more similar to the unvaccinated group than the fully vaccinated group. This is yet another way in which the study was not designed reliably.

The next problem is confounding factors (i.e., things other than the trait you are interested in that differ between your groups and have the potential to influence the results). You see, the chi-square tests that the authors used are quite simplistic, and they have no mechanism for dealing with confounding factors (which is, once again, why it is so important to randomize your samples rather than using convenience sampling). The things being tested are, however, quite complex. For many learning disabilities, for example, it is well known that they are affected (or at least tied to) race, sex, genetics, etc. Therefore, unless the two groups you are comparing are similar with regards to all of those factors, your tests aren’t valid. However, the authors gave no indication of matching their groups by race, sex, medical history, etc. Indeed, the authors even acknowledged that there were differences between the two groups with regards to at least some factors (like the use of medicines other than vaccines). Therefore, we can automatically chuck out all the comparisons that used “unadjusted data.” In other words, all the results in tables 2 and 3 are totally meaningless. All those comparisons between vaccinated and unvaccinated children are utterly worthless because the experiment was confounded and the authors didn’t account for that. So even if they had sampled randomly, used actual medical records, and controlled the type I error rate, those results would still be bogus.

Next, the authors move on to look specifically at “neurodevelopmental disorders (NDD),” but to do this, they combined children with ADHD, ASD, and any learning disability. That is not a valid, reliable way to do this, because those things are quite different from each other. You can’t just lump any learning disability in with ASD then going fishing for a common cause. They aren’t the same thing, and there is no reason to put them together.

Further, at this point their methods become really unclear. They say that they used, “logistic regression analyses,” but they don’t give any details about how the model was set up. There are a lot of assumptions that need to be met before you can use this method, and they don’t state whether or not those assumptions were met. Similarly, it is very easy to set up these models incorrectly, and they give almost no information about how theirs was constructed. I need to know things like whether they tested for multicollinearity, but that information isn’t given. Further, based on what little description they do give, it seems like they almost certainly over-fit the model by including meaningless categories like religion. Things get even worse from there, because they start talking about adjusting the model based on significant patterns, but they give no explanation of how they made those adjustments. A proper paper should not make you guess about how the statistics were done. In other words, when you read a paper, ask yourself the following question, “if I had their data set, could I use the information in the paper to exactly replicate their statistical tests?” If the answer is, “no,” then you have a problem, and you should be skeptical about the paper. The type of extremely terse description that is given in this paper is totally unacceptable.

that's not good enough memeNow, at this point, you might protest and argue that I am assuming that they set up the model incorrectly. That is, however, not what I am doing. I’m not saying that they did it wrong; rather, I am saying that I don’t know if they did it right. A good paper should describe the statistics in enough detail that I know exactly what they did, and this paper doesn’t do that. It does not make it possible for me to evaluate their methods. To put that another way, extraordinary claims require extraordinary evidence, and if you want to say that vaccines cause neurological disorders, then you are going to need some extraordinary evidence, and a paper where I have to assume that the authors knew what they were doing simply doesn’t cut it. It’s not good enough. Further, given all of the other problems with this paper, it seems pretty clear that the authors did not know what they were doing, so I am not at all willing to give them the benefit of the doubt when it comes to logistic regression.

Summary

In short, this paper is utterly terrible from start to finish. It used an extremely biased sampling design, it used an unreliable data collection method, and it used bogus statistical tests that were poorly explained and failed to control confounding factors and the type I error rate. Indeed, when you look at how this study was designed, it was set up in a biased way that almost guaranteed that it would find “evidence” that vaccines are dangerous. It would have been shocking if such a horribly designed study found anything else. This isn’t science, not even close. It is a junk study that very much deserved to be retracted.

Related posts

 

Posted in Nature of Science, Vaccines/Alternative Medicine | Tagged , , , | 10 Comments

Science matters because it works

Why should you support science? Because it works! It’s crazy to me that I even have to say that, but this is where we are as a society. Various forms and degrees of science denial are running rampant throughout our culture, and attacks on science are being disseminated from the highest levels. Indeed, it has gotten to the point that hundreds of thousands of scientists and science enthusiasts like myself feel compelled to take to the streets to march for science and remind everyone of the fundamental fact that science works and is unparalleled in its ability to inform us about reality and improve our world.

Image via the CDC

Just look around you. Everything that you see was brought to you by science. The batteries that power your electronic devices are a result of scientific advances in chemistry, as are the plastics that make up seemingly everything in our modern world. The planes that let you travel the world in mere hours were produced by our understanding of physics. The medicines that have doubled the human life expectancy came from biology, physiology, etc. Diseases that once claimed millions of lives each year are now almost unheard of thanks to advances in immunology, virology, etc. Even on topics where people frequently criticize science, like cancer, there have been great advances. Our ability to fight many cancers is improving, and, at the risk of appealing to anecdotes, I personally have family members who recovered from cancers that were untreatable just a few decades ago. Indeed, everyone reading this probably has friends and family who are only alive because of modern medicine (in fact, I would have died as an infant if it were not for medical technologies that my ancestors couldn’t dream of). Our entire modern world only exists because science works. Medicine, computers, cell phones, satellites, plastic, etc. all exist because science works. That is why it is so incredible to me that the anti-science movement even exists. Science has a proven track record, and we all benefit from it constantly.

Nevertheless, here we are, in a reality where the most powerful man in the world refers to an extremely well-established scientific fact as a “hoax,” a world where countless celebrities go around promoting all manner of unscientific woo, a world where opposing science has become a fashionable status symbol, a world where even a notion as ridiculous as believing that the earth is flat can enjoy a resurgence of popularity. This is crazy, and it has to stop. If we want to move forward as a society, or even just maintain our current position, we have to embrace science, not oppose it. Our views and policies have to match the facts, rather than trying to make the facts match our ideology.

Now, at this point, inevitably lots of people are going to get offended and respond with something to the effect of, “I’m not anti-science, but…I disagree with the way that science is being done, I think that massive corporations are buying off scientists, I have anecdotes that don’t match the science, scientists have been wrong in the past, scientists are close-minded, etc.,” but those aren’t valid responses and by using them you are standing in opposition to science, which makes you, by definition, anti-science. See, here’s the thing, science is a method, and it either works, or it doesn’t. You can’t pick and choose when you want to accept it and when you want to reject it. Either you accept that science is the only reliable method for understanding our universe that we have ever come up with, or you don’t.

This brings me to two important points. First, the people who make, “I am not anti-science but…” arguments are nearly always people with zero experience in science. They are people who are projecting their distorted preconceptions about science onto a method that they know nothing about. When people saying things like, “climate change scientists are just in in for the money” or “scientists are just going along with the dogma of their fields” they are just revealing how little they actually understand about how science operates or why we do research. No one gets funding for blindly going along with something that everyone already knows. You get funding for pushing boundaries and chasing novel ideas. Indeed, every great scientist was great precisely because they discredited the views of their day.

that's not how this works memeSecond, these arguments nearly always (I’m tempted to say always) arise when science conflicts with someone’s personal beliefs. For example, countless conservatives are happy enough to have science make more efficient batteries, predict tomorrow’s weather, cure their illnesses, etc., but the instant that it says that burning fossil fuels is bad, suddenly they turn on science and invent fanciful conspiracies, appeal to a minority of fringe researchers, cite discredited papers, etc. Conversely, droves of people stand behind the science of climate change 100%, but when exactly the same scientific method says that GMOs are safe, suddenly we are back in conspiracy land. That’s not how this works! You don’t get to oppose science just because it shatters your naïve ideology. When thousands of papers conducted by countless scientists from all over the planet arrive at the same conclusion, you don’t get to reject that conclusion just because you don’t like it.

A final group of dissidents take things even further and directly question the validity of science. They claim that decades of research on vaccines is discredited by the simplistic notion that “mothers know best.” They ignore the scientific impossibility of homeopathy in favor of personal anecdotes. They insist that the fact that something has been used for thousands of years is more important than the fact that numerous studies have shown that it’s nothing but a placebo, and they embrace all manner of utter nonsense about energy fields, crystals, resonant frequencies, etc.

All of this is, however, once again, discredited by the obvious fact that science works! We had anecdotes, appeals to antiquity/popularity/nature, maternal instincts, etc. for thousands of years, and they got us nowhere! Science is the thing that allowed us to tell which of those anecdotes were based on causal relationships and which ones were based on spurious correlations, and science is the thing that allowed us to know which natural remedies actually worked (e.g. aspirin) and which ones were hogwash. Further, science is the thing that let us improve on nature and synthesize purer and more concentrated forms of natural chemicals, as well as making medicines that aren’t even found in nature. Again, the evidence for this is everywhere! If you have diabetes and take insulin, for example, you get that insulin not from nature, but rather from a GMO that was produced by science. Similarly, if you need surgery, I’ll bet anything that you are going to want to be knocked out using the best anesthetic that science has to offer, rather than munching on some herbs. Again, science is a method, not a collection of information, and the method clearly works.

The history of tobacco actually illustrates this well. It was used medicinally for centuries by Native Americans, it was supported by countless anecdotes, it was 100% natural, mothers thought it was best for their children, etc. Today, however, we know that not only does it fail to cure illnesses, but it is extremely carcinogenic. Why do we know that? Because of science! Because careful and systematic studies revealed that all of those anecdotes, maternal instincts, etc. were wrong. Now, someone is surely about to write a comment about the time that scientists were paid off by Big Tobacco to support smoking or the doctors who thought smoking was safe, but those are distortions of history. Sure, there was a transition period when evidence was still being accumulated and scientists and doctors were not convinced (nothing in science changes overnight), but that didn’t last, because science prevailed. Similarly, yes, there were a minority of scientists that were paid off, and tobacco companies certainly put tons of money and effort into creating the illusion that there was no scientific evidence that smoking was dangerous, but that was entirely a smoke screen created by the companies, and, once again, their efforts ultimately failed.

Indeed, this is exactly the same thing that is happening today on so many issues. The science on climate change, for example, is extremely clear. It is supported by thousands of studies and is agreed upon by virtually all climatologists. Nevertheless, fossil fuel companies have done a marvelous job of creating the illusion of controversy. They have a handful of scientists that they publicize strongly, and they pour tons of money into promoting the notion that the science isn’t settled. The anti-vaccine movement is the same thing. The science is solid, but they have a handful of “experts” and pump so much money and effort into it that it appears that there is a conflict, even though this is a settled issue among medical experts. Similarly, big organic companies pump untold millions of dollars into opposing GMOs and making it appear that the science isn’t settled, even though nearly 2,000 studies have conclusively shown that GMOs are  safe for humans and no worse (or even better) for the environment than traditional crops. Finally, if you are prone to conspiracy theory musings, then consider this: massive, multi-billion dollar tobacco companies tried as hard as they could to buy of scientists and suppress the truth, but they utterly failed. So how likely do you really think it is that pharmaceutical companies, Monsanto, etc. succeeded at that endeavor?

This post became more of a rant than I had intended, so let me try to rein things back in for a concluding paragraph. My point is simple, fundamental, and obvious: science works. A huge portion of you are probably only alive today because of science, and even if you would have survived, statistically, many of your friends and family members wouldn’t have. Indeed, you benefit immeasurably from science every single day. The advances of the past roughly 150 years are incredible, and they only occurred because of the method known as science. I for one, want those advances to continue, and I certainly don’t want to go backwards, but that means supporting science. It means making it a priority, and it means accepting the results that it produces even when we don’t like them. I usually try not to get political on this blog, but the reality is that if you love the advances that science has produced, then you need to get political. You need to vote based on who supports science, and you need to tell your elected officials that it is not okay to reject science or to cut funding for it, and I’m not just talking about climate change here. If you want life-saving medical breakthroughs to continue, then you need to support funding for agencies like the NIH. If you want to benefit from an enhanced understanding of the universe, then you need to support funding for things like the NSF. If you understand how many technological wonders have come from the space program and want more technological advances, then you need to support funding for NASA. I could go on, but hopefully you get my point. The way that I see it, our society is at something of a crossroads, and either we will fight for science, support it, and move forward because of it, or we will reject it, downplay it, and ignore it, in which case, at best, we will stagnate and halt our progress, and at worst, we will move backwards (e.g., increased disease outbreaks as vaccination rates fall). The choice between those two options seems pretty obvious to me.

Related posts

 

 

Posted in Global Warming, GMO, Nature of Science, Vaccines/Alternative Medicine | Tagged , , , , , , , , , , | 10 Comments

The vaccine package insert paradox

The anti-vaccine movement presents a beautiful case-study in inconsistent reasoning and logical contradictions. One of the most entertaining and important of these contradictions comes from their treatment of vaccine package inserts. If you have ever spent any time debating anti-vaccers, then you have, no doubt, encountered these inserts. They list adverse events that were reported during vaccine testing, and anti-vaccers are adamant that these inserts provide clear evidence that vaccines are, in fact, dangerous. As I will explain, however, this argument is inconsistent with other core anti-vaccine arguments, and the presence of adverse events in the package inserts actually provides strong evidence against the vast global conspiracy that anti-vaccers envision (and, indeed, that their position requires).

blue stick figure white vaccine package insert paradox anti-vaccerFirst, it is important to clarify exactly what the adverse events on vaccine package inserts actually are, because anti-vaccers constantly get this wrong. They are not side effects that have been confirmed to be caused by vaccines. Rather, they simply include any adverse event that was reported during vaccine testing, regardless of whether or not the vaccine was the cause. For example, if, during testing, a child developed a fever from something completely unrelated to the vaccine, fever would still get listed as an adverse event (remember, saying “A happened before B, therefore A caused B” is a logical fallacy known as post hoc ergo propter hoc). In other words, the package inserts contain a lengthy list of anecdotes, and anecdotes are not evidence of causation. If you actually take a careful look at the lists, you should be able to convince yourself of this pretty easily. For one thing, many of the items on these lists (such as autism) have been carefully studied and repeatedly shown to have no causal relationship with vaccines. Further, the lists contain things like teething, and I doubt that even the most ardent anti-vaccer is willing to blame vaccines for teething (thus we get our first logical contradiction, because they are cherry-picking which adverse events to blame vaccines for). Finally, the package inserts themselves clearly state that the items on these lists have not been shown to be caused by vaccines. So, once again, anti-vaccers are being logical inconsistent because they are only believing the part of the package insert that agrees with their preconceptions. Just to prove that I’m not making this up, here is a quote from the Tripedia DTaP vaccine insert (an insert that anti-vaccers frequently cite; my emphasis).

“Events were included in this list because of the seriousness or frequency of reporting. Because these events are reported voluntarily from a population of uncertain size, it is not always possible to reliably estimate their frequencies or to establish a causal relationship to components of Tripedia vaccine.”

There is, however, a more serious contradiction, and it is the one that I want to focus on. Anti-vaccers insist that “big pharma” knows that vaccines are dangerous and are just covering it up for the sake of money. For that claim to actually work, however, you also need governmental regulatory agencies and pretty much all of the world’s scientists to be willing to cover up evidence of the dangers of vaccines as well. Thus, anti-vaccers are forced to concoct a vast conspiracy in which pharmaceutical companies lie constantly and have bought off the FDA, CDC, independent scientists, etc. Now, if all of that is true, then riddle me this, Batman, why would those lying companies publish a list of adverse events that has to be approved by the corrupt FDA?

Really think about this for a second. In the same breath, anti-vaccers will tell you that Big Pharma is lying to cover up the truth about vaccines and publishing a list that proves that vaccines are dangerous. Those two views are incompatible. If these companies are actually willing to buy off major government organizations and most of the world’s scientists, then why on earth would they undo all of that by publishing a list of harmful things that vaccines cause? (note: even though these lists don’t actually show causation, they are still clearly not in the pharmaceutical companies’ best interests, more on that in a minute)

You might try to worm your way out of this by arguing that Big Pharma doesn’t want to publish these lists, but the FDA forces them to. If that is your response, however, then you are correct that this is what is happening, but you are wrong that it helps your position. You see, if you argue that the FDA is forcing Big Pharma to do this, they you have just undercut the notion that Big Pharma bought off the FDA, and that is a huge problem for you. The FDA demands evidence that things are safe before it will approve them, so the only way that Big Pharma is going to be able to push “TOXIC” vaccines is if they have bought off the FDA, but if they bought off the FDA, then how is the FDA forcing them to publish these lists? Do you think that the FDA is ok with poisoning children just so long as the company prints a list of adverse events on the package insert? That makes no sense.    

A second approach would be to claim that companies are only publishing those lists to avoid lawsuits, but there are two problems with this argument. First, these lists are not standard warning lists, so it’s not actually clear to me that they would do much in a legal setting. Second, and more importantly, the National Vaccine Injury Compensation Program (VICP) was set up precisely so that people could seek compensation for “vaccine injuries” while protecting companies from expensive lawsuits. In other words, companies are already protected from lawsuits by the VICP, so they don’t need a list to do that (note: the VICP is essentially a no fault system that does not require evidence that the vaccine actually caused the injury, so it also doesn’t constitute evidence that vaccines are dangerous; details here).

Finally, you could try to get out of this mess with a shred of dignity by admitting that the lists don’t actually provide evidence that vaccines are dangerous. Although that is certainly the position that you should take, it actually doesn’t help you all that much, because a clear logical contradiction still remains. People respond incorrectly to labels all the time, and companies know this (that is why totally worthless labels like “organic” and “natural” are so common). Thus, even though these lists aren’t actually evidence against vaccines, people will (and clearly do) still view them that way, and pharmaceutical companies aren’t stupid. They know that people will miss-interpret those lists. Thus, publishing those lists is still bad for Big Pharma’s bottom line, which once brings us back to the question of why the companies publish them? I’m actually going to agree with anti-vaccers here, and agree that pharmaceutical companies would be more than happy to cover up anything that might hint that their products are dangerous. I’m under no delusions that pharmaceutical companies are benevolent entities setting out to bring about world peace and eternal youth. They are after money, plain and simple. Their greed is, however, kept in check by regulatory agencies like the FDA. In other words, we have once again arrived at the conclusion that companies publish these lists because the FDA requires them to do so. This is really important because, as I explained above, this is completely inconsistent with the notion that the FDA has been bought off by big companies, and if the FDA hasn’t been bought off by big companies, then you have no reason to think they aren’t actually doing their job and regulating pharmaceutical products.

To summarize all of this, the anti-vaccine movement relies on the notion that companies have bought off regulatory agencies like the FDA, because without that conspiracy, there is no explanation for the fact that agencies like the FDA approve vaccines. However, the FDA forces pharmaceutical companies to publish adverse events in the package inserts, even though doing so is bad for the companies. Herein lies the contradiction. If the FDA has been bought off, then how is it capable for forcing companies to publish these lists? These two things are incompatible with each other, and the fact that the FDA can force companies to publish these lists is clear evidence that the FDA controls the companies, not the other way around. Without a corrupt FDA, however, anti-vaccers’ conspiracy theory comes crashing down.

Posted in Vaccines/Alternative Medicine | Tagged , | 37 Comments

Scientists aren’t stupid, and science deniers are arrogant

Debating those who reject scientific facts has been a hobby of mine for several years now. It’s not a very rewarding hobby, and it comes with high stress levels and periodic fits of rage, so I don’t particularly recommend it. However, it has exposed me to countless pseudoscientific arguments on pretty much every topic you can imagine, and on each of those topics, I have found that not only do people with no formal training in science think that they know more than the entire scientific community, but in almost every case, they think that there is a fundamental and obvious problem that essentially all scientists have either missed or are willfully ignoring. If you think about this for a minute, it’s rather incredible. It’s amazingly arrogant to think that you can, via a few minutes of Googling, find a fundamental and obvious problem that essentially every scientist everywhere in the world missed, despite their years of training and experience. Nevertheless, that is exactly what most anti-scientists think (though they wouldn’t usually put it in those terms). Therefore, my intention is to provide several examples of this type of thinking using arguments from a variety of topics. Hopefully, this will illustrate the absurdity of this type of hubris and demonstrate the key point that I want you all to take home. Namely, if you think that you have found a simple and obvious problem that virtually every scientist on the planet missed, you are almost certainly wrong.

Note: Before I begin, I want to clarify that if you are one of the people who uses these arguments, I do not want you to think that I am attacking or belittling you. As I have previously written about, I used to be one of you. So, if you feel like I am making fun of you, realize that I am also describing my former self. To put that another way, I don’t think that you are stupid, but you are misinformed, and you are behaving irrationally.

This figure from Hansen et al. 2005 shows the effect of both the natural and anthropogenic drivers of climate change. Notice how only anthropogenic sources show a large warming trend. Also, see figure 2 of Meehl et al. 2004.

Let’s begin with climate change arguments. There are many that I could choose from here, but let’s start with the argument that the current warming is just a natural cycle because the climate has changed naturally in the past. If you like to use this argument, then I have several questions for you. Do you honestly think that climatologists never thought of this? Do you really think that the people who spend their lives collecting those data on past climates never even bothered to check and see if the current warming was part of a natural trend? I realize that I probably sound flippant here, but I’m actually asking these questions sincerely. Do you truly think that the entire scientific community is so hopelessly incompetent and stupid that they never even bothered to check the natural drivers of climate change? If you do, then I have news for you: they aren’t. Scientists have looked at past climate changes (Lorius et al. 1990; Tripati et al. 2009; Shakun et al. 2012), and they have very carefully looked at the natural drivers of climate change, and they have consistently found that the current warming does not match natural cycles and can only be explained by including our greenhouse gasses in the analyses (Stott et al. 2001; Meehl, et al. 2004; Allen et al. 2006; Wild et al. 2007; Lockwood and Frohlich 2007, 2008; Lean and Rind 2008; Foster and Rahmstorf 2011; Imbers et al. 2014).

A very similar argument proposes that the sun is the cause of climate change, and I have frequently encountered people who seem to truly think that scientists have never examined that possibility. Again, how stupid do you think scientists are? Do you really think that it never occurred to any of them that the giant nuclear furnace in the sky might be the problem!? News flash, it did. They’ve studied the sun’s output repeatedly and have consistently found that it is not the main driver of our current climate change (Meehl, et al. 2004; Wild et al. 2007; Lockwood and Frohlich 2007, 2008; Lean and Rind 2008; Imbers et al. 2014).

Similarly, climate contrarians love to point to the Antarctic sea ice and say, “the sea ice is increasing, so global warming can’t be happening.” Again, do you really think that scientists aren’t aware of that fact? Do you honestly think that thirty seconds on Google showed you a fact that the entire scientific community is ignorant of? Or, if they are aware of it, then what, are they just too stupid to comprehend it? There’s really only three possibilities here, and they are all nuts. To think that scientists have somehow missed this or are ignoring it, you have to think that all scientists are either stupid, hopelessly ignorant, or involved in some form of insane and enormous conspiracy. The more rational conclusion, however, is clearly that the situation must be more complex than a simple increase in sea ice would lead you to believe, and scientists must have information that you didn’t uncover via your degree from Google University. That is, of course, reality. For one thing, although Antarctic sea ice had increased (see note), ice shelves and glaciers globally are down and we keep setting new record highs for annual average temperature (WGMS 2013; Parkinson 2014; Stroeve et al. 2015). Further, when you look more closely at the situation with the Antarctic sea ice, you’ll find that it is being caused by a complex combination of factors including the levels of ozone in the atmosphere, shifting ocean currents due to ice melting elsewhere, etc. Gillett and Thompson 2002; Zhang 2007). My point is that these simple, obvious arguments almost never work. Reality is more complex than that.

Note: At the time of writing this, Antarctic sea ice was actually unusually low, but that is likely just a fluctuation, and it is too early to draw any solid conclusions. Thus, it may return to being high in the near future. Nevertheless, there is a very consistent global trend of decreasing ice.

we did not evolve from apes but we share a common ancestor with themClimate change deniers are, of course, not alone in their hubris. Creationists are right up there with them. Probably one of the most common examples of this type of flaw from creationists is the classic argument, “if humans evolved from apes, then why are there still apes?” As with the climate change arguments, I have to ask, do you honestly think that scientists are this stupid? Just think about this for a second. If this argument actually worked, then it would mean that basically every biologist for the past century missed an extremely obvious problem. To fully comprehend just how crazy that is, realize that we are talking about people who spent close to a decade receiving intensive training in biology and then spent the rest of their lives actually doing biology. You really think that in all of that they somehow missed the fact that humans and apes are both still around? Do you really believe that despite years studying every detail of fossils, constructing and comparing cladograms, etc. they never stopped to think about the fact that apes are still here? Do you honestly think that they are going to have their entire life’s work brought crashing down by a simple 11 word question? Does that seem rational to you? It shouldn’t. The reality is, of course, that this argument is a strawman fallacy. Evolution tells us that modern apes share a common ancestor with us, rather than us evolving from them. So, it is the creationists who missed something obvious and fundamental, not the scientists (to be fair, some creationist organizations do eschew this argument, but it is, nevertheless, common among the general public).

Another common creationist argument is the claim that evolution defies the laws of thermodynamics because those laws say that things constantly become more disorganized, whereas evolution says that things become more organized. Here again, do you honestly think the Google has endowed you with a better understanding of thermodynamics than people who spend their entire lives studying it? Do you really think that every scientist in the world is so fundamentally wrong about an extremely basic concept in science? And let’s be clear, here (and in the other arguments) you aren’t just saying that they are wrong, you are saying that they have all missed an extremely obvious, elementary problem that a high school student could see. That’s crazy. Once again, reality is far more rational, because reality tells us that systems are only required to become more disorganized when they are closed (i.e., when they aren’t receiving energy from other sources), but the earth is an open system (i.e., it gets energy from the sun) so things on it can, in fact, become more organized (that’s why trees can grow, you developed from a simple, single-celled zygote, etc.; details here). So, once again, it is the science deniers who are missing something obvious and fundamental, not the scientists.

Finally, this post certainly wouldn’t be complete without at least one example from anti-vaccers. I thought a lot about which argument to use here out of the many that I have to choose from, but I think for the sake of this post, the best example is the argument that vaccines aren’t effective because during many (but not all) outbreaks, most of the people who get the disease are vaccinated against it. As with every example that I have given, at a quick glance, it sounds like a really good argument. It seems like a slam dunk against vaccines, but if you think about that for five seconds, that should bother you. If this is such clear evidence that vaccines don’t work, then why haven’t any scientists or doctors paid attention to it? Why aren’t the people who publish these statistics concerned by them? Again, if you found this with a few minutes on Google, then why isn’t the scientific community aware of it? Or if they are aware of it, why don’t they care? The answer isn’t conspiracies, Big Pharma, or lizard-people. It’s simply that scientists are better at math than anti-vaccers are. It is true that in many ourbreaks (again not all), most infected people were vaccinated, but that is only because most people were vaccinated. When you look at the actual proportions, you consistently find that the disease rates were much higher among the unvaccinated. To give an analogy, most car accidents involve sober drivers, but that doesn’t mean that driving drunk is safer. Rather, it is a simple by-product of the fact that most people drive sober. When you look at the proportions, you find that the rates of car accidents are higher among drunk drivers.

I could continue to give many other examples both from these topics and pretty much every other “debated” topic in modern science, but I think it would be more profitable to spend the remainder of this post dealing with the counter arguments that I expect to receive. All of these are ones that I have dealt with in the past, so I will be brief here and will simply direct you to my other posts for more details.

First, you may be tempted to accuse me of an appeal to authority fallacy. However, there is a huge difference between appealing to authority and deferring to experts. I’m not saying that these things are true because scientists say that they are (that would be fallacious). Rather, I am trying to get you to engage in a simple exercise in plausibility. Ask yourself, does it honestly seem reasonable that untold millions of people with advanced degrees, years of training, years of experience, scores of publications, etc. missed something fundamental and obvious that you were able to find on Google? No, it doesn’t. Again, that doesn’t automatically make the scientists right, but it should make you very, very cautious about saying that they are wrong. It should give you great humility, and you should fact check extremely carefully using really good sources before you conclude that you are right and essentially every scientist in the world is wrong. To put that another way, you don’t need to be an expert to think that experts are right, but you do need to be an expert to think that they are wrong.

the fact that scientists wrong past conspiracy laughet atNext, you might try to say something like, “well, scientists have been wrong in the past” (debunked here) or “they laughed at Galileo, but he turned out to be right” (debunked here). There are numerous problems with this, so see my other posts for details, but I’ll give a Cliff Notes response here. First, scientific concepts have been discredited before, but they have always been discredited by other scientists doing real research. Second, most of the examples of scientists being wrong come from well before modern science even existed. If you limit yourself to the last 150 years or so (i.e. the age of modern science), you will find far fewer examples of a widely accepted concepts being discredited. Third, when those concepts were discredited, it wasn’t by some simple and obvious thing that everyone except for non-scientists were hopelessly ignorant of. It’s always been something complex or non-intuitive or usually both. It’s been something that had to be revealed by careful research, not an 11-word question. It’s never been something like scientists not bothering to check if the sun is driving climate change. To illustrate this, the Newtonian concept of gravity is one of the best examples of something that was widely accepted in the modern scientific era that turned out to be wrong, but that was discredited by the amazingly complex concept of relativity! Further, Newton wasn’t wrong so much as incomplete (as usually is the case).  Another good example is the concept that continents are stationary. This was replaced by plate tectonics, which, again, is neither obvious nor simple. Finally, it is worth mentioning that the days of a lone maverick operating outside of the norms of science are long gone. Modern science is an incredibly collaborative process, and new paradigm-altering conclusions come from teams of scientists with years of research, not someone sitting on their couch watching Youtube videos.

conspiracy theory skeptic science expertAt this point, you might contend that scientists are fully aware of these problems and are just covering them up for the sake of money, but that is insane. Arguing that essentially all of the world’s millions of scientists are involved in some sort of massive conspiracy is downright idiotic. I’ve talked about the math behind this before, but, in short, there is no motive in most cases (a lot of research is done by independent scientists), and just the sheer size of the conspiracy makes it implausible (it would have to involve every government, every health organization, every scientific body, and every university on the planet). Further, this argument is 100% an assumption. The burden of proof is on you to provide actual evidence that the world’s entire scientific community is corrupt, and unless you can do that, this is an ad hoc fallacy.

As a final attempt at a counter-argument, you might appeal to “dogma’ in science, and claim that there are scientists who see the problems but don’t speak out for fear of ridicule from their peers. This is, however, a complete misunderstanding of how science works. No scientist has ever been considered great for going along with the accepted wisdom of their day. Every great scientist was great precisely because they discredit the accept views of their day. To put that another way, for me personally, as a young biologist, nothing could possibly be better for my career than disproving evolution. If I actuaindiana jones fortune and glory kidlly had simple and compelling evidence that it was wrong, I could publish in any journal of my choosing, I would have my pick of universities to work at, and I would almost certainly receive a Nobel Prize. Disproving evolution would result in me going down in history as one of the great minds of the 21st century. So, why haven’t I or any of the thousands of other ambitious young biologists published that evidence? Because it doesn’t exist! This idea that you have to blindly go along with the “dogma” to get anywhere in science is totally backwards. You don’t get grants to confirm things that everyone already knows. Rather, you get grants, fame, and recognition for pushing boundaries, studying new ideas, and discrediting commonly held views. That’s how you achieve fortune and glory in science.

I find it baffling that so many people think that scientists are arrogant simply because scientists claim to know more about science than non-scientists.

My point in all of this is really quite simple. When you approach any scientific topic, you should do so with an appropriate amount of humility as well as an appropriate amount of respect for the fact that thousands of people spent their entire lives studying a topic that you are only learning about through Wikipedia. Anytime that an argument requires you to think that the entire scientific community is hopelessly stupid, ignorant, incompetent, etc. you should be extremely skeptical. Scientists aren’t stupid, and if you think you have found something simple and obvious that all of them have missed, you are almost certainly wrong. It is the epitome of arrogance to think that a few minutes or even hours on Google have endowed you with a better understanding of science than the collective scientific community gained through countless years of training and experience.

Literature Cited

  • Allen et al. 2006. Quantifying anthropogenic influence on recent near-surface temperature change. Surveys in Geophysics 27:491–544.
  • Foster and Rahmstorf 2011. Global temperature evolution 1979–2010. Environmental Research Letters 7:011002.
  • Gillett and Thompson 2002. Simulation of recent Southern Hemisphere climate change. Science 302:273–275.
  • Hansen et al. 2005. Earth’s energy imbalance: confirmation and implications. 308:1431–1435.
  • Imbers et al. 2014. Sensitivity of climate change detection and attribution to the characterization of internal climate variability. Journal of Climate 27:3477–3491.
  • Lean and Rind. 2008. How natural and anthropogenic influences alter global and regional surface temperatures: 1889 to 2006. Geophysical Research Letters 35:L18701.
  • Lockwood and Frohlich. 2007. Recently oppositely directed trends in solar climate forcings and the global mean surface air temperature. Proceedings of the National Academy of Sciences 463:2447–2460.
  • Lockwood and Frohlich. 2008. Recently oppositely directed trends in solar climate forcings and the global mean surface air temperature. II. Different reconstructions of the total solar irradiance variation and dependence on response time scale. Proceedings of the National Academy of Sciences 464:1367–1385.
  • Meehl, et al. 2004. Combinations of natural and anthropogenic forcings in the twentieth-century climate. Journal of Climate 17:3721–3727.
  • Parkinson. 2014. Global sea ice coverage from satellite data: annual cycle and 35-year trends. Journal of Climate 27:9377–9382.
  • Shakun et al. 2012. Global warming preceded by increasing carbon dioxide concentrations during the last deglaciation. Nature 484:49–54.
  • Stott et al. 2001. Attribution of twentieth century temperature change to natural and anthropogenic causes. Climate Dynamics17:1–21.
  • Stroeve et al 2012. The Arctic’s rapidly shrinking sea ice cover: a research synthesis. Climate Change 110:1005–1027.
  • Tripati et al. 2009. Coupling CO2 and ice sheet stability over major climate transitions of the last 20 million years. Science 326:1394–1397.
  • WGMS 2013. Glacier Mass Balance Bulletin. World Glacier Monitoring Service12.
  • Wild et al. 2007. Impact of global dimming and brightening on global warming. Geophysical Research Letters.
  • Zhang 2007. Increasing Antarctic sea ice under warming atmospheric and oceanic conditions. Journal of Climate 20:2515–2529.

 

Posted in Uncategorized | Tagged , , , , , | 168 Comments

The fallacy fallacy: Reject the argument not the conclusion

Two weeks ago, I wrote a post on the importance of understanding logical fallacies, and in that post, I made the following claim, “anytime that an argument contains a fallacy, that argument must be rejected.” Much to my surprise, many people took issue with this and brought up the fallacy fallacy (that’s not a typo). Some of those comments were simply pointing out the existence of the fallacy fallacy (which I actually did in the aforementioned post as well), but many of them were arguing that I was wrong or at least on shaky ground because of the fallacy fallacy. For example, one person said, “of course simply pointing out that someone’s argument is a fallacy is a fallacy in and of itself,” another said that although I was not committing a fallacy fallacy I was, “flirting with encouraging individuals to commit ‘the fallacy fallacy’” (those are exact quotes, not paraphrases). Thus, it appears that this topic may not be very well understood, so I want to spend this post talking about it, because it is an important concept to grasp. My original statement was correct and in no way misleading. Any time that an argument contains a logical fallacy, that argument is flawed and you must reject that argument. However, it is possible to have a flawed argument that still has a true conclusion. So, the fallacy fallacy only occurs when a bad argument leads you to reject the conclusion rather than the argument.

Logical fallacies are indeed disturbing.

As I explained in the previous post, deductive logical arguments should be set up such that if the premises are true, then the conclusion must also be true. In other words, the conclusion must follow necessarily from the premises (an argument with this property is known as a “valid” argument). However, logical fallacies often present an invalid logical structure in which the conclusion does not follow necessarily from the premises (in other cases they may operate by doing things like assuming false premises). Thus, logical fallacies are errors in reasoning and result in arguments that either aren’t valid or aren’t sound (a sound argument is one that is valid and has only true premises). Therefore, anytime that an argument contains a fallacy, the argument itself is flawed. The logical structure does not work, and you simply cannot use that argument in support of the conclusion. This is fundamental and vitally important to understand: you must always reject a flawed argument. If an argument contains a fallacy, then the argument does not work, and you cannot use it. However, that does not necessarily mean that the conclusion is false.

This is where fallacy fallacies come in. If you tell someone that their argument is wrong because it contains a fallacy, then you are adhering to the rules of logic and have not done anything wrong. However, if you tell them that their conclusion is wrong because the argument contains a logical fallacy, then you have committed a fallacy fallacy, because a bad argument tells you absolutely nothing about the conclusion.

Let me illustrate this using an example from the previous post. The following argument is not valid because it contains an affirming the consequent fallacy.

  • Premise 1: All men are mortals
  • Premise 2: Socrates is a mortal
  • Conclusion: Therefore, Socrates is a man

This is a bad argument. Because of the affirming the consequent fallacy, the conclusion does not follow necessarily from the premises (i.e., not all mortals are men). Thus, we must reject this argument. We simply cannot use this argument as a reason for thinking that Socrates is a man, but in this case, the conclusion is still true. Indeed, if you think about this, you should realize that it is always possible to construct a bad argument for a true conclusion. For example, I could say,

  • Premise 1: Aliens hate goats
  • Premise 2: Aliens like waffles
  • Conclusion: Therefore, the earth is spheroid

That argument is clearly nonsense. It doesn’t make the slightest bit of sense (it’s a non-sequitur fallacy) and both premises are rather bizarre assumptions, but the conclusion is still true! Nevertheless, although it is possible to have a bad argument and true conclusions, in many cases bad arguments do, in fact, lead to false conclusions (see previous post). In contrast, a sound logical argument guarantees that the conclusion is true. So, I reiterate that flawed arguments (including ones that contain logical fallacies) tell you nothing whatsoever about the conclusion. They provide you with absolutely zero evidence for or against it.

So, what does all of this mean practically for you? How should you deal with this in debates? Well, that really depends on whether or not the burden of proof is on you. Remember, the person making the claim is always responsible for providing evidence for that claim, whereas the other person is under no obligation to refute that claim (at least until actual evidence has been provided). So, let’s imagine first that you are not the one making the claim, and the burden of proof is on your opponent. Further, they claim that X is true because of argument Y (in other words, they are using argument Y to support conclusion X). However, you discover a logical fallacy in argument Y. At that point, you should point out that fallacy and reject argument Y, however, you should not make any claims about conclusion X without first introducing other evidence/arguments (more on that in a minute). In other words, the fact that argument Y is flawed tells you nothing about conclusion X, but because the burden of proof is not on you, you aren’t required to do anything else. They have to provide a new line of evidence/reasoning to demonstrate that conclusion X is true, and you are not obligated to accept X or take it seriously until they present that evidence.

Nevertheless, you may have evidence showing that conclusion X is in fact false, in which case, you are welcome to present that evidence and use it to refute X. In other words, saying “argument Y contains a fallacy, therefore conclusion X is false” is a fallacy fallacy, but there is absolutely nothing wrong with saying, “argument Y has a fallacy and, therefore, does not support conclusion X, however, we can tell that conclusion X is false because of argument/evidence Z.” In other words, you can (and indeed should) point out logical fallacies to demonstrate flaws in your opponents’ reasoning, but if you want to actually say that their conclusions are wrong (rather than simply that their arguments are wrong) then you have to present actual evidence to the contrary.

This brings me to the final scenario: situations where the burden of proof is on you. In these situations, you are making the claim and, therefore, it is your duty to present actual evidence. As such, if your opponent points out a logical fallacy in your argument, you must reject that argument and either present new evidence/reasoning or admit defeat. They are not obligated to disprove your conclusion, and you cannot continue to use the flawed argument. Thus, you are obligated to present a new, sound argument and real evidence in support of the conclusion.

In short, any time that an argument contains a logical fallacy, you must reject that argument. I stand by that initial claim. However, the presence of a fallacy (or other problem with the argument) tells you nothing about the conclusion. Therefore, you must always reject the argument, not the conclusion, otherwise you’re committing a fallacy fallacy. Further, to actually reject the conclusion, you need additional evidence/arguments that show the conclusion to be false.

Related posts

Posted in Rules of Logic | Tagged | 10 Comments

The importance of logical fallacies

From the Star Trek TOS episode “I, Mudd”

As anyone who frequents this blog knows, I spend a lot of time talking about logical fallacies. I frequently criticize peoples’ arguments for having them, and I present them as a reason for rejecting particular lines of thought. Nevertheless, many people fail to realize just how important they are, and showing someone that they have committed a fallacy rarely makes them reject their argument. Indeed, I once had someone say, “just because my argument technically contains a fallacy doesn’t mean that the underlying logic is wrong.” In reality, however, that is exactly what it means. Logical fallacies are, by definition, flawed lines of reasoning, and anytime that an argument contains a fallacy, that argument must be rejected. Therefore, understanding logical fallacies is critical for analyzing arguments and holding rational views, and in this post, I want to try to explain why fallacies are so important, how to detect them, and why their presence destroys an argument.

The structure of an argument

All arguments can be broken down into premises and conclusions. The premises are the facts that you are presenting, the conclusion is the thing that you are arguing for, and the goal is to set up the argument such that the conclusion must follow necessarily from the premises. In other words, for an argument to be a good argument, it must be set up such that if the premises are true, then the conclusion must also be true (this is what we call a “valid argument”). Additionally, the premises must, of course, actually be true (when both conditions are met, the argument is said to be “sound”). For now, I want to focus on the requirement that the conclusion must follow necessarily from the premises, but we will come back to the true premise requirement later.

 Note: I am talking specifically about deductive arguments here and throughout this post. There are other types (such as inductive and probabilistic) in which the premises show that the conclusion is most likely true, rather than that it must be true.

 To illustrate how this works, let me use the following example (this is set up in what is known as a syllogism).

  • Premise 1: Bill is larger than Bob
  • Premise 2: Bob is larger than Tom
  • Conclusion: Therefore, Bill is larger than Tom

This is a logically valid argument. In other words, as long as those premises are true, then the conclusion must also be true. There are no other options. If Bill is larger than Bob, and Bob is larger than Tom, then it must be true that Bill is larger than Tom (this example is also an illustration of something known as the law of transitive properties). Importantly, you should note that the underlying logical structure is what matters here. As long as that structure works (which it does), we can replace those premises with any other true premises, and the resulting conclusion will be true (as long as we haven’t changed the underlying structure). In other words, we can reduce this argument to the following logical structure:

  • Premise 1: A is larger than B
  • Premise 2: B is larger than C
  • Conclusion: Therefore, A is larger than C

Now, we can replace A, B, and C with any true facts, and the argument will work. For example,

  • Premise 1: Jupiter is larger than earth
  • Premise 2: The earth is larger than the moon
  • Conclusion: Therefore, Jupiter is larger than the moon

Or

  • Premise 1: A train is larger than an ant
  • Premise 2: An ant is larger than a bacterium
  • Conclusion: Therefore, a train is larger than a bacterium

I could keep going, but hopefully you get the point. It doesn’t matter what premises I use, or how disparate the items in them are. As long as the premises are true and I retain the same logical structure, then the conclusion must be true. Further, if you can find a single example in which this structure and true premises results in a demonstrably false conclusion, then you have shown that the argument’s structure must be flawed. In other words, for a deductive argument, the logical structure must work 100% of the time, or else the logical structure is flawed.

It may seem like I am off topic here, but understanding this is really important, because, as I will explain below, many logical fallacies operate by breaking an argument’s logical structure. In other words, they change the argument so that the conclusion does not follow necessarily from the premises.

Non-sequitur fallacies

Now that you understand the importance of a logical structure, let’s look at a large family of fallacies collectively known as non-sequitur fallacies. These occur anytime that an argument’s structure is such that the conclusion does not follow necessarily from the premises, but there are many specific subcategories and types of fallacies within that overarching umbrella term.

To begin, let’s look at what is probably the most common example in all of philosophy. Consider the following deductive argument:

  • Premise 1: All men are mortals
  • Premise 2: Socrates is a man
  • Conclusion: Therefore, Socrates is a mortal

We can reduce this argument to the following structure:

  • Premise 1: All X are Y
  • Premise 2: Z is X
  • Conclusion: Therefore, Z is Y

That may seem confusing, but if you think about it for a second, you should be able to convince yourself that it will work 100% of the time. If all X are Y, and Z is X, then Z must also be Y.

Now, consider the following extremely similar argument:

  • Premise 1: All men are mortals
  • Premise 2: Socrates is a mortal
  • Conclusion: Therefore, Socrates is a man

An example of an affirming the consequent fallacy

Now we have a problem. This argument does not work. The conclusion does not follow necessarily from the premises, and the reason for that is a logical fallacy known as affirming the consequent. This fallacy alters the logical structure in a way that prevents the premises from leading necessarily to the conclusion. We can write it as follows:

  • Premise 1: All X are Y
  • Premise 2: Z is Y
  • Conclusion: Therefore, Z is X

Again, if you think about that for a minute, you should see the problem. The fact that all X are Y does not mean that all Y are X. Thus, it is possible for Z to be Y, but not X. We can easily illustrate this with an example.

  • Premise 1: All men are mortals
  • Premise 2: My pet iguana is a mortal
  • Conclusion: Therefore, my pet iguana is a man

Obviously, that doesn’t work. It is clearly a bad argument. It has an invalid logical structure in which the conclusion does not follow necessarily from the premises, and, as a result, it produces an incorrect conclusion. Remember, if a deductive logical structure is valid, then it must produce true conclusions 100% the time (when supplied with true premises). Therefore, the fact that my example has an incorrect conclusion proves that this structure is invalid.

Now, what does this have to do with affirming the consequent fallacies? Well that name, “affirming the consequent” is simply the term that we use to describe this logical structure. In other words, by demonstrating that this logical structure is invalid, I have shown that an argument that contains this structure (i.e., that contains an affirming the consequent fallacy) is invalid. This is why it is so important to understand logical fallacies and take them seriously when they are pointed out to you: they result in arguments with invalid logical structures. In other words, they create arguments in which the truth of the premises does not guarantee the truth of the conclusion.

To further illustrate this, let’s move on from affirming the consequent fallacies and talk about a different fallacy: post hoc ergo propter hoc (or just “post hoc” for short). This is one of the most common fallacies that I encounter in debates about scientific topics, and it takes the following logical structure.

  • Premise 1: Q happened before U
  • Conclusion: Therefore, Q caused U

The problem with that should be pretty obvious: the fact that one thing happened before another doesn’t mean that one caused the other. In other words, the conclusion does not follow necessarily from the premise. We can easily illustrate this with simple examples.

  • Premise 1: I performed a sacrifice, then it rained
  • Conclusion: Therefore, my sacrifice caused the rain

Or

  • Premise 1: I read a book, then had a heart attack
  • Conclusion: Therefore, reading the book caused the heart attack

Do you see how that works (or, rather, doesn’t work)? The fact that one thing happened before another does not lead to the conclusion that there is a causal relationship. The logical structure is invalid, and any arguments containing this structure (i.e. containing a post hoc fallacy) must be rejected. On a side note, this is a fundamental reason why anecdotes are worthless as evidence of causation. The fact the you got better after taking something doesn’t mean that it worked, and the fact that you had an adverse event after taking something doesn’t mean that the treatment caused the event. Both of those arguments contain this structure (i.e., they are post hoc fallacies), and, as such, they are not valid, and the conclusion does not follow necessarily from the premise.

There are lots of other examples of this overarching type of fallacy, such as denying the antecedent, correlation fallacies, guilt by association, arguments from ignorance, etc., but they all have the same problem. Namely, they are invalid because they set up a logical structure in which the conclusion does not follow necessarily from the premises.

The fallacies of untrue premises

Another major “group” of fallacies work by either implicitly or explicitly making an untrue premise. The problem here should be obvious: if an argument relies on an untrue claim, then the argument must be rejected (i.e., it is not sound). As before, an easy way to test for this problem is to see if you can find any examples in which the argument doesn’t work.

Note: these groupings of fallacies are not officially recognized. They are just groupings that I personally find to be useful when thinking about fallacies and how/why they work (or don’t work, as the case may be).

Let me explain what I mean by using one of the most common variants of these fallacies: the appeal to nature fallacy. This fallacy occurs whenever someone asserts that something is good/useful/healthy because it is natural or that something is bad/useless/unhealthy because it is unnatural. When can set this argument up the following way.

  • Premise 1: X is natural
  • Conclusion: Therefore, X is good

That obviously doesn’t work, however, because there are plenty of true things that we can substitute for premise 1 that clearly result in false conclusions. For example:

  • Premise 1: The plague is natural
  • Conclusions: Therefore, the plague is good

Now, you could stop right there, and call this another variant of the non-sequitur fallacy, and you wouldn’t be wrong. This structure, as I have presented it, clearly is invalid because the conclusion does not follow from the premise. However, I think that there is a more useful way to think about this fallacy and others like it. Namely, this fallacy has an assumed premise that is false. It assumes that everything natural is good. Thus, there is really an implicit second premise.

  • Premise 1: X is natural
  • Premise 2: Everything natural is good
  • Conclusion: Therefore, X is good

That second premise is, however, clearly false, and as a result, the argument fails (i.e., it’s not sound). Importantly, that premise (or some variant thereof, including the inverse “everything unnatural is bad”) is present in all appeal to nature fallacies. Thus, anytime that this fallacy is present, the argument must be rejected, because it inherently assumes an untrue premise.

no matter what crackpot notion you believeThere are many other, “appeal to” fallacies, and they all have the same basic structure and problem. For example, appeal to authority fallacies occur when you say that something is true because of the person who says that it is true. When you do that, however, you are inherently invoking the premise that the person in question is infallible, which is clearly false. Other examples include appeals to popularity (which assume that everything popular is good/right), appeals to antiquity (which assume that anything old is good/right), appeals to tradition (which assume that anything traditional is good/right), etc. (note: the one exception to this structure is the appeal to emotion fallacy, which simply makes an argument based on emotions, rather than facts or logic).

Note: You could also apply my “implicit untrue premise” explanation to some of the non-sequitur fallacies that I described earlier. For example, you could say that post hoc ergo propter hoc fallacies include the assumed premise that if Q happens before U, then Q caused U. There is nothing wrong with that way of conceptualizing those fallacies, and you are welcome to use it, I just personally find that explanation to be more complicated when the premise isn’t as simple as “everything natural is good.” You can, however, think of these fallacies either way. You can think of them as having an implicit and untrue premise or as having an invalid structure. I don’t care which you use, just so that you understand the concepts.

Another common fallacy is much less subtle and directly states untrue premises. I am, of course, referring to the straw man fallacy. This occurs whenever you attack a weakened or misrepresented version of your opponent’s argument, then claim to have defeated their actual view.  In other words, you say, “My opponent believes X, and X is wrong for reasons Y” when, in reality, X is a distortion or misrepresentation of what your opponent believes. Thus, your first premise is false (there are also subsets of this fallacy such as reductio ad absurdum).

Fallacies of the false dilemma are yet another example of fallacies that operate via untrue premises. These take the form of “Either X or Y is true, X is false, therefore Y is true.” This sounds great, until you realize that premise one is false, and there was actually a third option (Z) that wasn’t stated.

Detecting logical fallacies

Finally, I want to briefly talk about some tools for detecting whether a logical fallacy has been committed. Obviously, your best bet is to study the different types of fallacies and learn how each of them works. I have compiled a list of common fallacies to help with that, as have many other sites (e.g. Internet Encyclopedia of Philosophy [this is probably the most comprehensive one], Skeptical Raptor, Your Logical Fallacy Is, and many others) . Let’s assume, however, that you don’t have time for that, you can’t be bothered, or maybe you have studied them, but still struggle with particular arguments (don’t worry, that happens to all of us). Fortunately, there are some simple things that you can do.

First, I strongly recommend that you practice breaking an argument down into a syllogism like I have done throughout this post (start with the actual facts in the argument). Often, when you do that the problems will jump right out at you. If nothing immediately jumps out at you however, then try replacing the facts with letters (again, like I have done throughout). Then, look carefully at that structure and see if it is valid. See if the conclusion has to follow from those premises, and see if there are any implicit premises that need to be added. If, at that point, it is clear that either the conclusion does not follow necessarily from the premises or that there is an implicit and untrue premise, then you are done. The argument is flawed and you should reject it. If neither of those are obvious, then move onto the next tool.

The second tool is simply to try to find examples where the logical structure of the argument fails. Use the syllogism that you constructed before, but this time, make actual premises that are true but unrelated to the topic of debate (like I did by using a sacrifice to show that post hoc fallacies were invalid). If you can find any examples (hypothetical or actual) where the premises are true, but the conclusion is clearly false, then you have just demonstrated that the logical structure is invalid (assuming that you were careful and did not alter the structure, otherwise you’ve committed a straw man fallacy). This is a very useful tactic that you should get in the habit of using (I explained it in more detail here).

Although those two tools are useful, unfortunately, they aren’t all-encompassing. There are many other types of fallacies that I have not covered here because they are more specialized and difficult to generalize. Many of these are actually errors in debate tactics more than errors in reasoning. For example, a red herring fallacy occurs when, in a debate, you ignore your opponent’s argument/question and go off on an irrelevant side tangent in order to dodge a problem that they pointed out (politicians are masters of this). This type of fallacy is much harder to detect via a simple key like what I have presented, because there is no way to really construct a syllogism. It’s not a proper argument. Rather, it is a means of avoiding an argument. Similarly, for both straw man fallacies and false dilemma fallacies, you need to have enough knowledge on the topic at hand to tell that a false premise has been presented. That is the only way to detect them. So, although the tools that I have presented are useful and work in many situations, there really is no substitute for actually studying fallacies and becoming familiar with them.

Conclusion

Obviously, this post has been far from exhaustive, and there are many other fallacies (and even types of fallacies) that I didn’t address. However, this should give you a basic understanding of why fallacies are a problem, as well as some tools for detecting them. Anytime that a fallacy is present, the argument must be rejected, because you cannot be confident that the conclusion is actually supported by that argument. Thus, you should be mindful of logical fallacies and strive to avoid them in your arguments and views. Further, if someone points out that you have committed a fallacy, take that accusation seriously and look closely at their claim to see if it is correct. No one is immune to these flaws in reasoning, but there is no excuse for ignoring them once they have been pointed out to you.

 Note: It is worth emphasizing that when an argument contains a logical fallacy you must reject the argument not the conclusion (rejecting the conclusion rather than the argument is actually a fallacy known as the fallacy fallacy [that’s not a typo]). It is entirely possible to have an invalid argument, but a true conclusion. In other words, your conclusion may be true, but you cannot use that particular argument to support it, and it must be supported by other lines of evidence/reasoning.

Related posts

Posted in Rules of Logic | Tagged , , , , , , , , , , | 19 Comments

Vaccine injuries and confirmation biases

Humans are remarkably bad at accurately discerning patterns. All of us (skeptics included) are prone to confirmation biases and logical flaws in reasoning. We don’t like to be wrong, and, as a result, we tend to cling to things that appear to support our preconceived notions, while blindly ignoring things that discredit our views. Even worse, we tend to seek out others who share our views, and we enter echo chambers where we only hear from people who agree with us. This can happen for any of our views, whether they are views on science, politics, religion, etc. and it sets up a dangerous cycle, wherein we first settle on a position, then only accept information that agrees with that position, while simultaneously allowing that information to reinforce our original position. It’s a circle of ignorance that constantly bolsters our view, even if that view is totally incorrect. Again, this can happen to anyone on any topic, but in this post, I want to briefly use anti-vaccers and “vaccine injuries” as an instructive example in how this plays out.

Image via Things Anti-vaccers Say. Also, note that this whole line of reasoning is absurd. DNA doesn’t work the way that they are proposing.

If you ask people who oppose vaccines why they do so, you will likely receive a litany of “vaccine injuries” that they have personally seen or heard reported by others. Some of the more well-known of these include things like autism and SIDs, but the list generally doesn’t stop there. Depending on which anti-vaccer you ask, you may also hear that vaccines cause asthma, allergies, depression, violence, lupus, fibromyalgia, behavioral ticks, shaken baby syndrome, and just about every other issue that you can think off. Indeed, anti-vaccers seem content to view almost everything as a vaccine injury (even homosexuality [again, depending on who you ask]). The Facebook page Things Anti-vaccers Say does a good job of documenting these assertions, and I will include several of their screenshots throughout this post as both examples and discussion points.

It’s important to pause here to note several things. First, to be clear, vaccines do have side effects (as do all real medical interventions), but those side effects are either extremely mild or extremely rare, and the known benefits far outweigh the risks. Further, most of the things that anti-vaccers blame vaccines for simply are not caused by vaccines. For example, the association between vaccines and autism has been well-studied, and those studies have consistently found that vaccines do not cause autism (details and sources here). Similarly, not only do vaccines not cause SIDs, but they may actually reduce the rate of SIDs (details and sources here [see #64]). This raises the obvious question of why anti-vaccers see vaccine injuries everywhere that they look?

See this post for sources and more details.

Part of the answer is simply a reliance on shoddy sources. Anti-vaccers frequently get their information from notoriously counterfactual websites like Natural News, Green Med Info, Info Wars, and a host of others. Further, they readily accept anecdotes as evidence, which is where the topic of “vaccine injuries” really comes into play. What I see happening over and over again is that one parent tells a tale of how their child received a vaccine then developed autism, started coughing, died from SIDs or shaken baby syndrome, etc., and everyone in the anti-vaccine community makes an astronomical and logically invalid leap to the conclusion that the vaccine was responsible. In reality, however, anecdotes cannot establish causation, and the fact that one event followed another does not mean that the first event caused the second one. Indeed, that line of reasoning is a logical fallacy known as post hoc ergo propter hoc. Things do sometimes just happen to occur together. For example, in a previous post, I looked that the rate of autism and the rate of vaccination and showed that, mathematically, we would expect there to be thousands of cases each year where the first signs of autism were noticed shortly after a vaccination just by chance! This is a major reason why anecdotes aren’t valid evidence: coincidences do happen. Further, using anecdotes for things like vaccines and autism is particularly absurd because of the overwhelming scientific evidence that vaccines do not cause autism. You don’t get to reject a scientific study based on an anecdote. That isn’t how science works.

Note: Many anti-vaccers also site things like VAERS, package inserts, and the VICP, but those are also unacceptable sources for establishing causation as I explained here.

The explanation that I have just offered seems good at first, but it really just raises more questions than it answers. Most importantly, why are anti-vaccers willing to accept shoddy evidence? This is where confirmation biases come in, and I’d like to use the screenshot to the right as an example. When most people hear a report like this of a “healthy 100-year-old woman dying in six months” (paraphrasing) the cause of death seems pretty obvious: she was 100 years old! It is well established that the very elderly have weakened immune systems and  a reduced ability to recover from illnesses and injuries, and even someone who is in good health (for a 100-year-old) can succumb very quickly to even a minor aliment. Nevertheless, as you can see in the comments, to some anti-vaccers, this had to be a vaccine injury! She received a vaccine (or so they assume) then died a few months later, so the vaccine had to be the cause! (See what I mean about anecdotes being worthless as evidence of causation?)

I can, of course, give lots of other examples of this, and some are even worse. For example, when faced with evidence that homeopathic teething tablets were potentially causing sickness and even deaths in young children, the anti-vaccer on the left chose to blame vaccines instead. Think about how incredible this is. Here, you have the FDA saying that they have evidence showing that this product is likely dangerous, but instead of listening to the FDA, this anti-vaccer shifted the blame to vaccines despite a complete and total lack of evidence to support that claim! They were so convinced of the evil of vaccines that they were willing to blindly overlook an actual danger and place the blame on vaccines instead.

So, what is going on here? This answer is confirmation biases and, more generally, motivated reasoning. Anti-vaccers believe strongly that vaccines are dangerous, and that belief causes them to see vaccine injuries everywhere. That’s what confirmation biases do: they make you latch onto anything that seems to support your position, even if the evidence is shoddy. Thus, when most of us hear about something like a 100-year-old woman who “suddenly died,” we realize that there are lots of potential causes that are far more plausible than a vaccine, but if you are a committed anti-vaccer, you don’t see those other causes, because they don’t fit your mental narrative. All that you see is the vaccine. Similarly, for the homeopathic anti-vaccer, accepting that a homeopathic product was dangerous didn’t fit their mental narrative, but thinking that vaccines were at fault did fit their mental narrative. So, despite the evidence suggesting that the homeopathic product was dangerous and despite the total lack of evidence to suggest that vaccines were the cause, they blamed the vaccines. That is how powerful confirmation biases and motivated reasoning can be. They can make you think that something dangerous is actually safe and that something safe is actually dangerous.

confirmation biases

Although this image is targeted specifically at anti-vaccers, this circle of ignorance can occur for anyone on any topic. Even if your conclusion is correct, it may bias you and cause you to both accept shoddy sources and use those sources to bolster your position.

Now that we have gone over a few examples, let’s turn to what I think is the most dangerous aspect of confirmation biases: their ability to reinforce a belief. Every time that an anti-vaccer sees a non-existent vaccine injury, it reinforces their belief that vaccines are dangerous, which is obviously a big problem. This goes back to the circle of ignorance that I mentioned earlier. For one reason or another, anti-vaccers decided that vaccines are dangerous. Because of that belief, they see “vaccine injuries” that aren’t really there, and those “vaccine injuries” then make them even more convinced that vaccines are dangerous. It is a never-ending circle that sinks them deeper and deeper into ignorance and conspiracy theories. Further, this situation is made even worse by online groups where anti-vaccers can see the anecdotes of other anti-vaccers. Here again, their confirmation biases cause them to blindly accept these stories as evidence against vaccines, even though it is illogical to do so.

At this point, you may be ready to laugh at anti-vaccers for being stupid, but you shouldn’t be. As I have previously argued, most people who reject science aren’t stupid or crazy, they are just misinformed and have succumbed to their confirmation biases and motivated reasoning. Importantly, this is not a problem that is limited to science-deniers. It is a problem that affects everyone. All of us are prone to confirmation biases, and we all should constantly check our views to make sure that we are following facts and logic rather than following what we want to be true. We need to carefully consider evidence rather than blindly latching onto the things that fit our mental narratives. Perhaps most importantly, we should always consider the possibility that we are wrong. Being willing to be wrong is (in my opinion) the only way to break the circle of ignorance. Unless you are actually willing to be wrong, you will never be able to accept evidence that is contrary to your beliefs, and it is important that you are capable of doing that. To quote an excellent Dr. Who episode, “the circle must be broken.”

Note: Please do not use the VAERS, package inserts, or VICP as evidence unless you have read this post.

Note: You may be wondering how a homeopathic tablet could be dangerous since they are usually just water, so here is the story in a nutshell. This product is supposed to contain very tiny amounts of belladonna, a substance which is quite harmful in higher doses (as is true of many active ingredients in homeopathy). However, the dose makes the poison, so when diluted to extremely low levels (as homeopathic products generally are) this product is harmless (it’s also useless, but that’s beside the point). However, the FDA kept receiving numerous reports of children becoming sick or even dying after taking the tablets, and the symptoms matched the known symptoms of belladonna poisoning. So, the FDA started investigating, and they found that the amount of belladonna in the tablets was inconsistent, which is a huge problem because, as I said, this chemical is toxic at higher doses. So, it is extremely important that it only be present in very low doses. Thus, although this does not conclusive demonstrate that the homeopathic tablets were at fault, the combination of inconsistent doses of a potentially toxic chemical and the fact that children who took it where experiencing the symptoms of receiving a high dose of that chemical certainly makes a good case against the homeopathic product. Again, a more proper study is necessary for a conclusive answer, but the evidence is certainly strong enough that parents should take it seriously, and blindly rejecting that evidence in favor of arbitrarily blaming vaccines is insane. Finally, please note that this is a very different situation from the anecdotes that parents use against vaccines, because those anecdotes are all over the map, don’t match the symptoms of vaccine ingredients at the doses at which they are present, and frequently fly in the face of actual studies. In most cases, there is no plausible mechanism through which the vaccine could be at fault.

Posted in Rules of Logic, Vaccines/Alternative Medicine | Tagged , , , , , , , | 4 Comments