The Logic of Science

A few days ago, I posted what I thought was a fairly innocuous image (right) onto my blog’s Facebook page. I was, however, sadly mistaken. My page was quickly flooded with comments by people who arrogantly insisted that there was nothing wrong with blindly rejecting all of the thousands of studies showing that vaccines are safe. I probably shouldn’t have been surprised by this, but still, I was astounded by the level of hubris and willful ignorance that was being so proudly displayed. What didn’t surprise me, however, were the attempts at justifying such a baffling position. They included all of the usual tropes about conspiracies, scientists being paid off, government corruption, etc. (I have included screen shots of some of the responses to the meme throughout this post). Most of these responses suffered the same fundamental problem. Namely, they assumed that there was something wrong with the studies rather than actually providing evidence that they were flawed. This is a very common mistake. When faced with a study that disagrees with their preconceptions, people often blindly assert that the authors were paid off, the data were manipulated, there’s a conspiracy afoot, etc., but unless they can actually prove that such unethical behavior occurred, that response is logically illegitimate and is no different from simply saying, “it’s wrong because I say it’s wrong” (in technical terms, it’s an ad hoc fallacy). You cannot assume that a study is flawed just because you disagree with it.

To be clear, I am not suggesting that you blindly accept every scientific paper. Unfortunately, not all papers are of a high quality, and bad, biased research does get published. So you should carefully examine all scientific studies, but you cannot blindly reject them just because they discredit your preconceived views. This is especially true for topics like vaccines for which we have thousands of studies that all agree with each other. It’s one thing to say that one paper is biased or one research group was paid off, but it is something else entirely to assert that essentially every paper on a topic is wrong and every researcher is corrupt.

In short, it is important to carefully examine scientific studies rather than blindly accepting them, but the inverse is also true (i.e., you have to carefully examine the study before rejecting it). I have previously written about good criteria for rejecting a paper; therefore, in this post I want to flip things and instead describe 12 bad arguments for rejecting a paper.

Bad reason #1: Galileo/Columbus

When faced with results that they don’t like, many people will invoke Galileo or Columbus and claim that they defied the mainstream view and people thought that they were crazy, but they turned out to be right. I explained this one in detail here, but to be brief, no one thought that Galileo was crazy. He presented facts and careful observations, not conspiracies and conjecture. He did not blindly reject the science of his day, rather he made meticulous observations and presented data that discredited the common views. That is not in any way shape or form the same as arrogantly and ignorantly rejecting a paper just because you disagree with it.

Moving on to Columbus, the debate in his day was about the size of the earth, not its shape, and Columbus was dead wrong. In fact, his stubborn ignorance would probably have killed him and his crews if it hadn’t been for the fortunate “discovery” of the New World. You see, Columbus was the ideological equivalent of a modern day anti-vaccer. He “did his own research” and pompously declared that all of the experts were wrong about the size of the earth, and he thought that the well-accepted calculations were flawed. In reality, the accepted calculations of his day were very close to correct, while Columbus’s numbers were way off.

Bad reason #2: science has been wrong in the past

Moving beyond the specific examples of Galileo and Columbus, other people often make the broad claim that science shouldn’t be trusted because it has been wrong before. This is another one that I have dealt with at length elsewhere, so I’ll be brief here. First, it is true that science has been wrong, but it has always been other scientists who have figured out that it was wrong. Further, it is logically invalid to blindly assume that it is wrong just because it has been wrong before.

Additionally, although there have been plenty of minor hypotheses which have been discredited, there have been very few core ideas that have been rejected in the past century. In other words, ideas which are supported by thousands of studies have rarely been rejected, and very few central ideas have been overthrown in recent decades. The closest example that you can find is probably Einstein’s theory of relativity replacing Newton’s law of gravity, but even in that example, Newton’s work wasn’t really wrong, it was just incomplete. Einstein didn’t completely throw Newton out the window; rather, he simply showed that Newton’s law doesn’t always work and doesn’t give us a complete picture. He built on what Newton had found.

Finally, attacking science by asserting that it has been wrong before is utterly absurd because science is inherently a process of modifying our understanding of the world. In other words, science is self correcting. This is one of it’s greatest strengths. To publish in science, you have to pass a rigorous peer-review process, which weeds out a ton of junk science. So, most of what gets published is of high quality. Further, when a bad paper gets published, it quickly comes under scrutiny by the rest of the scientific community, who will then point out errors in it (if they exist) and often try to replicate the results. As a consequence, it can be hard to get away with fraudulent science because if someone else tries to replicate your work, they are going to figure out that something was very wrong with your research (this is exactly what happened with Wakefield’s fraudulent paper suggesting that vaccines cause autism). Thus, science is self correcting and constantly replaces erroneous ideas as new evidence comes to light (the same can’t be said for anti-science views which rigidly cling to their positions no matter how much evidence opposes them). Therefore, the fact that science has been wrong is actually a good thing, because if there were no instances where we had discovered that a previous idea was wrong, that would mean that science hadn’t advanced.

Bad reason #3: it’s all about the money

This is probably the most common response to papers on climate change, vaccines, GMOs, etc., and it’s often simply untrue. The scientific community is massive, and there are thousands of independent scientists doing research. Further, all scientific publications require authors to declare any conflicts of interest, so you can actually check and see if a paper was paid for by a major company, and if you did that, you would find that many of the papers supporting GMOs, vaccines, etc. have no conflicts of interest. Anti-scientists, of course, have no interest in actually looking at the paper. They would rather just assume that it was paid off because that fits with their world-view. Further, even if 90% of the papers on a topic like vaccines had been paid off, that would still leaves us with hundreds of papers showing that they are safe and effective and essentially no papers saying that they are dangerous (you can find more details on the finances of vaccines, GMOs, and climate change here).

Based on this person’s follow up comments, everything except for the first sentence was sarcastic (i.e., they think that science does actually blindly reject answers we don’t like, etc.)

Finally, even if a paper does have a conflict of interest, that doesn’t give you carte blanche to ignore it. The fact that someone works for a pharmaceutical company, for example, does not automatically mean that they biased or falsified their data. If a paper has a conflict of interest, then you should certainly give it extra scrutiny, and you should be suspicious if it disagrees with other papers or has questionable statistics, but you cannot automatically assume that it is flawed.

Bad reason #4: there are other results that I disagree with

This is one of my favorites. Someone will say, “I reject the science of X because science also says Y and I disagree with Y.” We can rephrase this as, “I reject science because I reject science.” I would not, for example, accept water fluoridation as evidence that it’s ok to reject the science of vaccines unless I had already rejected the science of fluoridation. In other words, you have to justify your rejection of the science of Y before you can use it as evidence that we shouldn’t trust the science of X. Further, even if you could demonstrate that the science of Y (in this example fluoridation) was wrong, that still would not in any way shape or form prove that the science of X (in this example vaccines) is wrong. In fact, this entire line of reasoning is just a special case of the logical fallacy known as guilt by association. If are going to say that a scientific result is incorrect, you have to provide actual evidence that the specific result that you are talking about is incorrect.

I’m not sure which part of this post grieves me the most: their shoddy logic, their rejection of science, their bizarre capitalization, or the fact that someone “liked” it.

Bad reason #5: gut feelings/parental instincts

I encounter this one frequently, and it irritates me to no end. I will, for example, show someone the scientific evidence for vaccines, and they respond with, “well as a parent only I know what is best for my child.” Similarly, when I show people the evidence for GMOs, they often respond with something like, “well I just have a gut feeling that manipulating genes is bad.” I do not give a flying crap about your instincts or gut feelings. The entire reason that we do science is because instincts and feelings are unreliable. When someone presents you with a carefully conducted, properly controlled study, you absolutely cannot reject it just because you have a gut feeling that it’s wrong. Doing that makes no sense whatsoever. It is the most blatant form of willful ignorance imaginable. Don’t get me wrong, intuition is a good thing, and gut feelings can certainly help you in many situations, but they are not an accurate way to determine scientific facts.

Just to demonstrate the true absurdity of this response, let’s imagine for a minute that you went into the ER, and the doctor there said, “according to scientific studies, I should only give you X amount of morphine, but my gut tells me that I should actually give you five times that amount, so that’s what I’m going to do.” I’m pretty sure that you would immediately demand a different doctor. Similarly, imagine someone saying, “science says that smoking causes cancer, but my gut tells me that it’s fine.” Do you see the problem? Gut feelings simply aren’t reliable. That’s why we do science.

Bad reason #6: I’m entitled to my opinion/belief

This is another very common response, and it is very similar to #5. Science deals with facts, not opinions or beliefs. When multiple scientific studies all agree that X is correct, it is no longer a matter of opinion. If you think that X is incorrect, that’s not your opinion, you’re just wrong. Think about the relationship between smoking and lung cancer again. What if someone said, “well everyone is entitled to their opinion, and my opinion is that it’s safe.” Do you see the problem? Scientists don’t have an opinion or belief that smoking is dangerous; rather, it is a scientific fact that it is dangerous, and if you think that it is safe, you are simply in denial. Similarly, you don’t get to have an “opinion” that the earth is young, or vaccines don’t work, or climate change isn’t true, or GMOs are dangerous, etc. All of those topics have been rigorously tested and the tests have yielded consistent results. It is a fact that we are changing the climate, a fact that vaccines work, a fact that the earth is old, etc. If you reject those, you are expressing willful ignorance, not an opinion or belief.

Bad reason #7: I’ve done my research/an expert agrees with me

I’ll make this one simple: if your “research” disagrees with properly conducted, carefully controlled studies, then your research is wrong (or at the very least, must be rejected pending future data). There, it’s that simple. The only exception would be if your research is actually a large set of properly controlled studies which have directly refuted the study in question (e.g., if you have a meta-analysis vs. a single study, then, all else being equal, go with the meta-analysis). It’s also worth pointing out that having a few people with advanced degrees on your side does not justify your position (that’s a logically fallacy known as an appeal to authority). No matter what crackpot position you believe, you can find someone somewhere with an advanced degree who thinks you’re right.

Bad reason #8: scientific dogma

This response basically states that all scientists are forced to follow the “dogma” of their fields, and anyone who dares to question that dogma is quickly ridiculed and silenced. I’ve written about this before, so I’ll be brief here. In short, that’s simply not how science works. Nothing makes a scientist happier than discovering that something that we thought was true is actually false. In fact, that is how you make a name for yourself in science. No one was ever considered a great scientist for simply agreeing with everything that we already knew. Rather, the great scientists are the ones who have shown that our current understanding is wrong and a different paradigm provides a better understanding of the universe. To be clear, if you are going to defeat a well established idea, you are going to have to have some very strong evidence. After all, “extraordinary claims require extraordinary evidence,” but if you have that extraordinary evidence, then you absolutely can publish it. If, for example, I actually had powerful evidence that discredited the theory of evolution, not only could I publish in the journal of my choosing, but I would have just guaranteed myself the Nobel Prize. As a biologist, nothing could possibly be better for my career than discrediting Darwin. So why then aren’t biologists rushing to publish that evidence? Quite simply, because it doesn’t exist. Similarly, you don’t see many publications against anthropogenic climate change, vaccines, etc. because the data for those positions just don’t exist (fun fact: “data” is plural so “the data don’t” is actually grammatically correct).

Bad reason #9: distrust of governments/media

I often find that people reject science because of a distrust of governments or the media. For example, anti-vaccers often blindly reject all CDC statistics showing the benefits of vaccines (amusingly they readily accept the reported side-effects, inconsistent reasoning anyone?). Many people, however, take it even a step further. On numerous occasions, I have shown someone a study which was not in anyway affiliated with a government agency, yet they still responded with a lengthy rant about corrupt governments or the media. The basic idea of their argument seems to boil down to, “the government/media agree with these results, therefore they must be false.” This line of reasoning is, however, clearly fallacious (in fact it’s a logically fallacy known as guilt by association). Governments and the media will lie to push their own agendas, I’m certainly not denying that, but that fact does not automatically mean that everything that they say is a lie. For example, the CDC and other government agencies say that smoking is dangerous, does that mean that it’s safe? Obviously not. Similarly, if a news reporter said that you shouldn’t drink lava, would that mean that you should? It’s fine to be skeptical of what you are told by the government/media. In fact it is a good thing, but when you are presented with scientific evidence, then it’s not a matter of trusting the government/media. Rather, it is a matter of whether or not you accept science. In other words, I don’t need to trust the government or media in order to accept the results of a carefully controlled study.

This comment was a bizarre combination of distrust of the government, doing one’s own research, following a personal belief, and confusingly trying to equate science with government decision in order to assert that mistakes have been made in the past (or at least that’s my understanding of this ramble)

Bad reason #10: it’s a conspiracy

This one is very closely related to #8 and 9, but it takes things a step further. It proposes that there is a massive conspiracy and scientists are being paid by governments/big companies to falsify results. Just take a quick look at the anti-vaccine movement or the anti-GMO movement, and you will quickly find that pro-vaccine/pro-GMO scientists are vilified and receive constant accusations of being “shills.” Similarly, there are many people who think that all climate scientists have been bought off by governments. I’ve explained the problems with this line of reasoning in more detail here and here, so I’ll just talk about the biggest problem. Namely, the scope of this conspiracy would be impossibly huge. The scientific community consists of millions of people from all over the world working out of thousands of universities, institutes, non-profits, corporations, agencies, etc. It includes people from countless religions, cultures, political ideologies, etc. There is no way that you could possibly get that many people to agree on a massive deception like this. Just think about what is being proposed here. Do you honestly think that nearly all of the world’s climate scientists have been bought off? We are talking about thousands of people from all over the world. Similarly, there are numerous corporations, universities, non-profits, etc. involved in the research and production of vaccines and GMOs. Do you honestly think that all of those different organizations (many of whom compete with each other and have different goals and purposes) have all managed to come together to make one unified conspiracy? That’s just nuts. The same problems exist for governments. Topics like vaccines, GMOs, and the dangerous of climate change are agreed upon by numerous governments and scientific organizations from all over the world. Honestly ask yourself the following question: which is more plausible, that countless governments, companies, non-profits, etc. have all come together to create the world’s largest conspiracy and buy off virtually every scientist on the planet, or that the thousands of independent scientists who have devoted their lives to science are actually doing real research?

Bad reason #11: anecdotes

This list certainly wouldn’t be complete without talking about personal anecdotes. I can’t even begin to tell you how many times I have presented someone with scientific studies showing that vaccines are safe only to have them responded with, “but I know someone who developed autism after getting a vaccine” or “what about this case where someone became sick after a vaccination.” Anecdotes do not matter in science, because anecdotes don’t allow us to establish causation. Let me give an example. Suppose that someone takes treatment X and has a heart attack 5 minutes later. Can we conclude from that anecdote that treatment X causes heart attacks? NO! It is entirely possible that the heart attack was totally unrelated to the treatment and they just happened to coincide with one another. Indeed, I once heard a doctor describe a time where he was preparing to vaccinate a child, and while preparing the vaccine, the child began having a seizure (to be clear, he hadn’t vaccinate the child yet). He realized that if he had given the vaccine just 60 seconds earlier, it would have looked for all the world like the vaccine had caused the seizure when in fact the kid just happened to have a seizure at the same time that a vaccine was being administered.

From those two examples, it should be clear that anecdotes are worthless because they cannot establish causal relationships (in technical terms, using them to establish causation is a logical fallacy known as post hoc ergo propter hoc fallacies [i.e., A happened before B, therefore A caused B]). Properly controlled studies, however, do allow us to establish causation. If, we took a large group of individuals of the same age, ethnicity, medical history, etc., divided them randomly into two groups, and gave half of them treatment X and half of them a placebo, then and only then would we be able to look for causal relationships. In other words, if the treatment group has heart attacks significantly more frequently than the control group, then we could conclude that treatment X most likely causes heart attacks (science never proves anything with 100% certainty). Nothing else will let us make that claim. Even if you collected a whole series of anecdotes in which people had heart attacks following treatment X, it wouldn’t matter because there wouldn’t be any controls. In other words, I could respond to your anecdotes with anecdotes of people who received treatment X and didn’t have heart attacks as well as anecdotes of people who didn’t have treatment X, but still had heart attacks. Properly controlled studies are the only way to establish that one thing causes another. That goes for side effects of vaccines, alternative “medicines,” fad diets, etc.

Bad reason #12: a scientific study found that most scientific studies are wrong

This argument is fascinatingly ironic because it uses a scientific paper to say that we shouldn’t trust scientific papers, but let’s look closer because this argument actually has some merit. The paper being references is, “Why most published research findings are false” by John Ioannidis, and it is actually a very useful and informative work, but it often gets misused. The paper describes several reasons why published papers are often wrong, and I will go over just a few highlights. First, we have the problem of small sample sizes. As I have previously explained, small sample sizes are unreliable and you really need a large sample size to be confident in your results, yet many studies get published with small sample sizes, and you should be hesitant to place a lot of confidence in a result that didn’t come from a good sample size.

Second, we have publication bias. This can be a bias because of funding sources or preconceived ideas, but often it is a bias that is inherent in the publication system. In science, it is (unfortunately) often hard to publish a “negative” result. For example, if you do a drug trial and you find that it doesn’t work, you may have trouble publishing that result; whereas, if you got a “positive” result (i.e., it does work) you could easily publish it. The problem is that statistical significance relies on probabilities, and some papers will, get a false positive just by chance (this is called a type I error and I explained it in more detail here). So, when journals only publish positive results, you end up with a lot of false positives which aren’t balanced out by the negatives, because the negatives don’t get published. In other words, the type I error rate among published papers is much higher than the rate among all studies, because negative studies often don’t get published.

Now, all of that may sound very bleak, but it should not make you lose all confidence in the scientific process because of a very important component of scientific inquiry: replication. Ioannidis’s work applies mostly to single paper studies. In other words, when only one study has ever looked at drug X, there is a high chance that the results are actually wrong, but when multiple studies have tested drug X and all found that it works, then you can be fairly certain that it is actually effective. So, the arguments set forth by Ioannidis don’t apply to topics like vaccines, GMOs, and other areas of “settled science,” because they have been examined by thousands of studies. When numerous studies all agree, then you can have very high confidence in the results (this is why meta-analyses and systematic reviews are so useful). So, this paper shouldn’t make you question the safety of vaccines, the effects we are having on the climate, etc. It should, however, make you skeptical of the one or two anti-vaccine papers that you occasionally see, or the one paper supporting some “miracle cure,” or the occasional paper on homeopathy, acupuncture, etc. Those studies almost always have tiny sample sizes and countless other studies have failed to replicate their results. This is why it is so important to look at the entire body of literature not just a single study.

Conclusion

In summary, properly conducted, carefully controlled studies are the only way to reliably understand our universe, and you cannot reject them without good justification. Look around you. All of the modern marvels that you see today were brought to you courteously of science. Further, if I asked you, “How many of your siblings died of a terrible childhood disease?” I’m guessing that the answer would be “none.” If I had asked that question a few decades ago, however, most of you would have lost at least one sibling to diseases which are now almost unheard of. Even if you want to erroneously attribute the decline of those diseases to increased sanitation rather than vaccines and modern medicine, it is still science which is responsible for our increased hygiene and access to clean water. So no matter how you cut it, many of you wouldn’t be alive today if it wasn’t for science. Science clearly works and you need an extremely strong justification for rejecting scientific results.

To be fair, some scientists are corrupt and bad science does occasionally get published, but bad research tends to be identified and discredited by other researchers. In other words, there may be a high probability of a single paper being wrong, but when lots of different studies have all arrived at the same conclusion, you can be very confident in that conclusion. Perhaps most importantly, you cannot simply assume that a paper is bad just because you disagree with its results. You need to present actual evidence that it is flawed or biased before you can reject it.