Amusingly, anti-scientists (such as anti-vaccers) always claim to have the upper hand on scientific knowledge. I have yet to meet one who hasn’t claimed to be “well-informed” or to have “done their research.” Yet when you ask anti-vaccers for their sources, you invariably get links to blogs and websites like Whale.to and NaturalNews.com, and those of us who dare to assert that we should be getting our information from carefully controlled studies, not blogs, are generally accused of having been “brainwashed,” “misinformed,” or “indoctrinated.” I almost never see an anti-scientist produce a legitimate source in support of their position. Therefore, in this post, I am going to use an analogy to demonstrate why there is one and only one way to know whether or not something works and is safe.

Suppose that someone came out with a product (product X) that you apply to multiple areas of your car and, according to the manufacturers, in 95% of vehicles it will make them last 100,000 miles longer than they would without it. This product is inexpensive, but, according to the manufactures, in 0.03% of vehicles, there will be a very slight reduction in fuel mileage, and in 0.000003% of vehicles, it will either cause a problem that will need to be repaired or, in extremely rare cases, it will destroy your vehicle. Now, you want to know with a high degree of certainty whether or not the manufacturer’s claims are true (after all, the life of your vehicle is at stake). How do to you test their claims? Please actually answer this question for yourself before reading any further, how would you determine with a high degree of certainty whether or not they are correct?

One of the most convenient options is to ask your friends and see what their experiences with the product have been, but this is obviously problematic. Suppose you have a friend who didn’t use it and has had his car for 300,000 miles, does that mean the product isn’t necessary because his car is just fine without it? No, it could just be dumb “luck” that his car is still running. Also, what? if you have another friend who used product X and has driven her car 300,000 miles. Does that mean that the product works and is safe? No, because her experience also could be due to chance or any number of other factors. Finally, you have a third friend who used it and their car died within 1,000 miles. Does that prove that it is bad for your car and the company lied? No, maybe they were just one of the ones who were unfortunate enough to be in the 0.000003%. For that matter, we can’t even be certain that product X caused the problem. Assuming that product X caused the problem is a post hoc ergo propter hoc fallacy (i.e., A preceded B, therefore A cause B). This could be one of the 5% of cases where product X simply didn’t work, and the car just happened to die after using X for reasons that were totally unrelated to X. The point is that polling your friends obviously doesn’t work because it is all anecdotal. There is no way to go from scattered personal accounts to a definite answer.

Because your friends can’t help, you then decide you use the internet. Surely by expanding your sample to encompass people’s comments on the internet you can find the answer. On the internet, however, you find the same problems that you had with polling your friends. You find lots of people giving their personal experiences and opinions, but, once again, there is no way to say that their experiences weren’t from chance. Also, the internet is notoriously untrustworthy. Anyone can write a blog about this product even if they know nothing about it. Further, for every blog in favor of product X, you find another one against it. There are multiple blogs and forums where people rant against the product and claim either that it is a conspiracy by the government to kill older vehicles and get them off the road, or it is just a scam by the manufacturer to make money. The same people also refer to themselves with appealing terms like “thinkers,” and they claim that everyone else has been brainwashed or indoctrinated to believe what the manufacture has told them. This all sounds legitimate, but how do you actually know that this group of people is correct? Further, you find plenty of other blogs that say the exact opposite, and both sets of blogs claim to have the facts and evidence. How do you tell which ones to trust? You obviously need to fact check both sides, but this becomes problematic because the “facts” all seem to be coming either from anecdotal evidence like what your friends gave you, or are just made up and are really no more than opinions. Once again, getting a definitive answer is impossible.

Finally, in frustration over the lack of good information online, you turn to your local mechanics and ask them what they think. Most of them say it works, but a few have reservations about it. A mechanic is obviously a better source of information than your non-mechanic friends or the error-prone internet, but still you cannot accept a mechanics word as proof (that would be an appeal to authority fallacy). Sure, they know cars very well, and they have actually used product X, but ultimately, they are giving their opinions about anecdotal evidence, and their opinions can be biased by any number of factors. Humans are notoriously bad at accurately seeing trends without the aid of statistics. Our minds are wired to look for patterns, but that often causes us to see patterns that don’t exist. So, some subtle bias that your mechanic has may cause him to inadvertently think that the product is working more often than it is or, inversely, that it is damaging vehicles more often than it is. Further, there may be a bias in the shops clientele. Perhaps most of the customers at this shop drive high end vehicles that generally have a long life span, so to the mechanics at this shop it seems like the product works because most vehicles that they see have high millage. A different shop, however, attracts customers who drive their vehicles very hard, so most of the vehicles that the mechanics there see have low millage and are falling apart, making them conclude that product X doesn’t work. Finally, you have the issue of which mechanics to believe. Do you just blindly accept the majority? Do you go with the ones that you personally like more? Do you toss a coin? None of those options result in a definitive answer.

At this point, I think that we can all agree that there is one and only one way to tell with a high degree of certainty whether or not the manufacturer’s claims about product X are true. We take an extremely large number of vehicles and carefully control for make, model, year, driving conditions, etc. Then we randomly choose half of them and apply product X. Meanwhile, the other half receives an inert dummy product. We then track the state of these cars over many years, and we record how many of the cars with product X need repairs and compare that with the repair rates on the control cars. Similarly, were compare the total life spans of cars with and without product X. Ideally, multiple different people would do this test multiple times so that we have several very large data sets. Then, we look at the data. If the product works and is safe for cars, then in all of the data sets, we should see that on average, cars with product X last longer than cars without it, and the damages that product X supposedly causes occur just as frequently in both groups. On the other hand, if product X is actually dangerous, we should see that vehicles that used it needed repaired more frequently than vehicles that didn’t use it. Only then, after doing a carefully controlled, randomized study can you conclude with a high degree of certainty that product X does or does not work. This is, of course, not simply my opinion. It should be intuitively obvious that an actual experiment is the only way to know, and any statistics book or professor will tell you that the only way to infer causation is to control all of the confounding variables so that only the experimental variables remain.

So what is my point in all of this? This situation is completely analogous to vaccines, alternative medicines, etc. If you agreed with me that the controlled study was the only reliable source of information (as any reasonable person would) then you must agree that carefully controlled studies are the only way to know with a high degree of certainty whether or not vaccines work and are safe as well as whether or not alternative medicines actually work. This means that if you are going to actually be well-informed about these you cannot trust blogs and personal stories. You must read the actual, original peer-reviewed papers where the results of the research are reported. Blogs are inherently second hand information. Even when they claim to be discussing scientific results, they often insert their own biases and distort the results. You must read the original literature because it is the only legitimate source of scientific information. You cannot trust anecdotes, and you cannot trust the internet. If you are getting your information from HealthNews and similar sites, you are not well-informed, it’s that simple.

In conclusion, lets apply this logic to vaccines:

1. If vaccines cause autism (or cancer or auto-immune diseases or any of the other problems that they are accused of), then people who get vaccinated should have those side effects more often than people who don’t get vaccinated.
2. Multiple studies have shown that unvaccinated people have these problems just as frequently as vaccinated people
3. Therefore, vaccines do not cause these problems

It doesn’t matter how many people claim that vaccines gave them autism or some other deformity, the carefully controlled studies clearly show that they are wrong.