Las
t week, I wrote a post on the hierarchy of scientific evidence which included the figure to the right. In that post, I explained why some types of scientific papers produced more robust results than others. Some people, however, took issue with that and accused me of committing a genetic fallacy because I was attacking the source of their information rather than the information itself. They were specifically unhappy about my claim that personal anecdotes, gut feelings, counter-factual websites, etc. did not constitute scientific evidence. After all, how dare I assert that their opinions weren’t as valuable as a carefully controlled study (note the immense sarcasm). In reality, of course, my argument was not fallacious, and they were simply misunderstanding how the genetic fallacy works. This misunderstanding is, however, quite common and somewhat understandable. The genetic fallacy can admittedly be very confusing. Therefore, I want to briefly explain what this fallacy is, how to spot it, and when it is and is not acceptable to criticize the source of an argument/piece of information.
If you’re a regular reader of this blog, then much of this may sound very familiar. That is because I have already covered a lot of the key points in a previous post on ad hominem fallacies. The ad hominem fallacy is generally considered to be a type of genetic fallacy; therefore, the same general rules apply.
Note: in this post, I am going to specifically deal with this fallacy as it pertains to scientific issues.
What is the genetic fallacy?
As it’s name suggests, the genetic fallacy results from attacking the source or origin of information, rather than the information itself. If you think about that for a second, the reason for the confusion becomes clear. On the one hand, the reason that genetic fallacies don’t work is obvious: the truth of a claim is not dependent on the one who is making the claim. Even someone who is wrong 99.9% of the time will occasionally be right. On the other hand, however, the source of the information is clearly important. It’s intuitively obvious that not all sources are equal, and some sources are more authoritative than others. Imagine, for example, that during a trial, the prosecution brought in some random guy off of the street and asked him to testify about the forensic evidence of the case. The defense would very correctly attack the source of that information by arguing that this person was not a credentialed expert and, therefore, his testimony should not be trusted. There is obviously nothing fallacious about that, and the prosecution clearly couldn’t respond by accusing the defense of a genetic fallacy (they also couldn’t respond by saying “well he watched some Youtube videos on crime scene investigations and he’s read some blogs and done thousands of hours of research”).
So how do we resolve this apparent dilemma? The answer is that attacking the source of a claim is only fallacious if the source is irrelevant to veracity and trustworthiness of that claim. The Internet Encyclopedia of Philosophy defines it like this (my emphasis):
A critic uses the Genetic Fallacy if the critic attempts to discredit or support a claim or an argument because of its origin (genesis) when such an appeal to origins is irrelevant.
In other words, there is nothing wrong with attacking a source, if the source of the information is actually germane to whether or not you should trust the information. So, if someone cites questionable sources like Youtube videos or personal anecdotes, there is nothing wrong with you saying that we shouldn’t trust that information, because the sources actually are unreliable. That’s no different from not trusting some random guy off the street as an expert witness in a courtroom. Remember, that the burden of proof is always on the person making the claim, so it is their responsibility to provide you with evidence from a trustworthy source. As a result, if they make a claim like, “vaccines are dangerous” and their “evidence” is an Info Wars article, you are under no obligation to discredit that article. Rather, it is their obligation to provide you with evidence from a reliable source.
It’s important to note, however, that you can only use attacks against a source to show that the information cannot be trusted. You cannot use them to say that the information is false. For example, if someone presents you with “evidence” from a Natural News article, there is nothing wrong with saying, “Natural News is not a reliable source, therefore we should not trust that information.” It would, however, be fallacious to say, “Natural News is not a reliable source, therefore that information is wrong” (technically that would be a special case of the fallacy fallacy). Even an extremely unreliable source may be right every once in a while.
In addition to assaults on the source of the information, the genetic fallacy can also occur when you attack the reason for a person holding a particular view. For example, I frequently see creationists attack their opponents by saying, “you only accept evolution because you are an atheist who doesn’t want to believe in God.” Even if that premise was true (which it often isn’t), it’s irrelevant. It has no bearing on whether or not evolution is true, and is, therefore, a genetic fallacy.
Finally, it’s important to realize that for an argument to be a genetic fallacy the assault on the source has to actually be the argument. For example, if you show me a scientific study, and I respond by saying, “well the authors of that study are just ugly idiots so I don’t need to listen to them,” then I would have committed a genetic fallacy (specifically, an ad hominem fallacy). If, however, I explained at length why the study was flawed, then concluded with a Trump-like jab at the authors appearance/intelligence, I would not have committed a fallacy. It would be uncouth and inappropriate for me to do that, but it wouldn’t actually be a fallacy because the attack on the source was tangential to my argument.
Addendum (19-Jan-16): The genetic fallacy also occurs if you assert that something is true because of its source (i.e., the appeal to authority fallacy is actually a type of genetic fallacy), but in this post, my focus was on attacking sources, rather than using them as proof of a position.
The genetic fallacy vs. the hierarchy of scientific evidence
Now that you understand what this fallacy is, let’s bring it to bear on the topic that inspired this post: the hierarchy of scientific evidence. It should by now be clear that using the hierarchy of evidence to assess the validity of a scientific claim is not the same thing as committing a genetic fallacy. Nevertheless, let’s look closer.
First, let’s look at my assertion that personal opinions, anecdotes, anti-science websites, etc. do not count as scientific evidence. It’s worth noting, that I didn’t actually say that they aren’t trustworthy. Rather, I simply said that they aren’t scientific evidence, and that claim is demonstrably true because those sources do not produce evidence via the standards and methodologies of science. Therefore, they are, by definition, not scientific evidence. If I ask someone to give me scientific evidence for a position, then I am asking for actual original research. I want to see the peer-reviewed paper that found the result that they are reporting, not the Youtube video they watched.
This flowchart summarizes the steps required to publish a peer-reviewed paper and the steps required to publish a blog post. Take a careful look at this difference, then honestly tell me that you think that blogs are a better source of information about science (more details here).
Nevertheless, although I didn’t claim that non-scientific sources are untrustworthy in the original post, I clearly think that they are. People often take issue with this, but if you stop and think about it for a second, the claim is self-evident. All that I am saying is that for scientific topics, we have to use scientific evidence, which necessarily comes from the peer-reviewed literature. Websites, Youtube videos, etc. are inherently second hand information, which may or may not be reliable. The scientific literature, on the other hand, is primary information. When you read a scientific paper, you can see the actual results of an experiment rather than simply reading someone’s biased explanation of those results. Further, to publish a peer-reviewed paper takes a tremendous amount of work. You have to pass a rigorous peer-review process during which numerous other scientists will evaluate your work to ensure that it was done correctly. In contrast, any idiot with a computer and internet connection can make a website/Youtube video with absolutely no assurance of quality control. To be clear, that doesn’t automatically mean that the information contained in second-hand sources is wrong, but it does mean that you don’t have any reason to trust that information, which is why they aren’t valid sources for scientific topics. Further, websites like Natural News, Info Wars, Answers in Genesis, etc. are notorious for containing inaccurate information, which gives you an extremely strong, relevant, and legitimate reason not to trust them.
Even within the scientific literature, however, you should be looking critically at the sources. Some experimental designs are simply more powerful than others and produce more reliable results. For example, if you have a meta-analysis of randomized controlled trials vs. a cross sectional analysis, it would not be a genetic fallacy to say that the cross sectional analysis is less reliable than the meta-analysis. From a strictly mathematical point of view, cross sectional studies are weak. They simply cannot make causal conclusions. In contrast, randomized controlled trials are very powerful and can make causal conclusions, and meta-analyses are even better because they combine multiple data sets, thus greatly increasing the sample size and reducing the chance of reaching a faulty conclusion. It’s a simple mathematical fact that meta-analyses are better than cross sectional analyses. Therefore, the type of study (i.e., the source of the information) is extremely relevant to the trustworthiness of a study, and using that information in a debate does not constitute a genetic fallacy.
Note: this flowchart only works when you are making an attack. Appeals to authority are also a type of genetic fallacy which I did not cover in this post or flowchart (you can find an explanation of them here).
Conclusion
Genetic fallacies occur when you make an irrelevant attack on the source of information rather than the information itself. That does not mean, however, that it is always fallacious to attack the source of information. Some sources clearly are better than others, and the burden of proof is always on the person making the claim. Thus, it is their responsibility to provide high quality sources, and you are not responsible for disproving the information from extremely low quality sources. Nevertheless, determining when attacks on sources are fallacious can admittedly be confusing. Therefore, I have constructed the flowchart on the right to help you determine when you can and cannot attack a source.
Note: Just to be clear, arbitrarily accusing someone of being a shill without providing actual evidence that they are being paid off does not constitute a legitimate, relevant concern.
More posts on logical fallacies:
- The Rules of Logic Part 3: Logical Fallacies
- The Rules of Logic Part 5: Occam’s Razor and the Burden of Proof
- The Rules of Logic Part 6: Appealing to Authority vs. Deferring to Experts
- Stop accusing me of ad hominem fallacies you stupid idiots
The Logic of Science 
This massively overgeneralizes. For instance, a large cross sectional study is often better than a small controlled trial. The claim that cross sectional studies are mathematically weaker than controlled trials is simply false in the context where you use it. Controlled trials strongly imply causality, but can only search for effects that are already believed to exist. Cross sectional studies can discover any effects visible in the data, and generally have massively larger sample sizes. Under many reasonable priors, a large cross sectional study is even stronger at implying causality than a weak controlled trial. Over reliance on controlled trials is a major factor holding back scientific advancement today.
LikeLiked by 1 person
I would like to clarify several points here. First, in this post, I was admittedly very general about the hierarchy of evidence, but that is because this post was focused on the genetic fallacy, not the hierarchy. In the initial post on the hierarchy of evidence (linked at the beginning) I talked about exceptions to the hierarchy, and discussed the importance of sample size and the fact that it can trump the hierarchy in some situations. Thus, when I claim that randomized controlled trials are more powerful than cross sectional analyses, I am talking about the methods themselves, not specific examples. In other words, I’m saying that if they have the same sample sizes, randomized controlled trials are more robust.
Second, it is true that cross sectional studies can be used to look for possible patterns, while randomized controlled trials can only test patterns that are already suspected (again, I explained that at length in the previous post), but that is an entirely different situation than comparing power. When we talk about comparing the power of two studies, they both have to be after the same question. In other words, both studies need to be looking for relationships between the same two variables.
Third, and perhaps most importantly, cross sectional studies cannot imply causality, even with a large sample size. They can show strong associations, but they simply can’t show causation. They generally can’t show whether the supposed cause preceded the supposed effect, and they can’t randomize, which makes it very difficult to account for all confounding factors. This is why randomized controlled trials are so powerful: they can account for many variables by carefully selecting the candidates, and they can randomize across the variables that they can’t control. One of the fundamental tenants of statistics is the importance of randomization, which is why randomized controlled studies are in fact mathematically more robust.
I explained all of this in more detail in the previous post, and there is an extensive amount of literature on the topic (there are some good books/papers to get you started at the end of the post).
LikeLiked by 1 person
“Under many reasonable priors, a large cross sectional study is even stronger at implying causality than a weak controlled trial.”
Interesting, how does that work? I can see the evidence for association would be stronger but causality?
LikeLike
The author wrote “It’s a simple mathematical fact that meta-analyses are better than cross sectional analysis” and does not imply what you’re accusing him of.
LikeLike
“For example, I frequently see creationists attack their opponents by saying, “you only accept evolution because you are an atheist who doesn’t want to believe in God.” ”
This might not always be the genetic fallacy, as your flow chart suggests it could be a clumsy accusation of confirmation bias, if the evidence use of the opponent is under attack.
“First, let’s look at my assertion that personal opinions, anecdotes, anti-science websites, etc. do not count as scientific evidence. It’s worth noting, that I didn’t actually say that they aren’t trustworthy. Rather, I simply said that they aren’t scientific evidence, and that claim is demonstrably true because those sources do not produce evidence via the standards and methodologies of science.”
Careful Fallacy man, you are engaging in your dangerous generalizations again! It is demonstrably true that anecdotes/personal opinion *can* produce evidence by the methodologies of science. Would you really discount as non-scientific a report of pain in a clinical trial or a report of the side effect of a drug or in the past reports of meteorites, rogue waves or ball lightening or the entire field of phenology? Perhaps such stuff is less reliable than other types of data but it is not necessarily non-scientific. Even personal opinion can be systematised to reach scientific conclusions, check out the field of “expert elicitation” although even I reckon this is on the the limits of what you can do with “quasi-data”. Plus ultra.
Re: meta-analysis. Yes it is powerful but if there is a bias in the considered studies then there could be a problem.
LikeLike
In my creationist example, what I meant was that the creationist is using that attack rather than dealing with the evidence that they are being presented (i.e., the scientist presents evidence, and they respond with “you’re an atheist”). That may not have been entirely clear in the post though.
I would contend that a report of pain in a clinical trial is not an anecdote, rather it is a data point. The difference between data and anecdotes is that data are collected in a careful and systematic way, whereas anecdotes are not. In a clinical trial, you can actually do something with reports of pain because you can compare reports among the two treatment groups. Outside of a clinical trial, however, if someone reports pain after taking a drug, it is virtually impossible to assign causation. There are no controls and no reference points to compare to, which is why I say that they are not scientific evidence. Science is a process of carefully and systematically studying the universe. An anecdote, however, is by definition not systematic and therefore cannot be construed as science. Again, that’s not to say that you can’t use those anecdotes as the starting point for scientific investigation, but the anecdotes themselves are not scientific evidence.
Your point about meta-analyses is very true, and I talked about that in the original post on the hierarchy of evidence.
LikeLike
OK, I would very much agree with you that systematisation is a core of science and I think your comment “The difference between data and anecdotes is that data are collected in a careful and systematic way, whereas anecdotes are not.” is really interesting. This begs an interesting question about whether an “anecdote” is not anecdotal in itself but how it is subsequently treated. You would then agree the plural of anecdote can be indeed be data then?
LikeLike
Yes and no. Let me use an example to illustrate what I mean. Let’s say that a mother vaccinates her child, and he/she develops autism. She then uses that as evidence that vaccines cause autism. That’s clearly an anecdote, and I think that you and I would both agree that it does not actually constitute scientific evidence that vaccines cause autism. At the same time, there are of course lots of other parents with the same story, as well as lots of children who were vaccinated and didn’t develop autism, were not vaccinated and did develop autism, etc. Each one of those independently is an anecdote, and in isolation none of them tell us anything even remotely reliable about whether or not vaccines cause autism. To circumvent that, I often see anti-vaccers making collections of anecdotes, where, for example, they poll other anti-vaccine parents. So in that situation, data clearly isn’t the plural of anecdotes, because the anecdotes were collected in a biased, non-systematic, and invalid manner.
Now let’s say that instead of combining the anecdotes that way, a researcher sets up a systematic case-controlled study. At that movement, each anecdote ceases to be an anecdote and becomes a point in a body of data. Nevertheless, if you were to rip those points out of the context of the systematic study, they would revert to being anecdotes.
In other words, I am arguing that whether or not something is an anecdote is determined by the context and manner in which it is collected. So anecdotes can become data if they are complied in a careful systematic way, but I’m not really comfortable with describing data sets as collections of anecdotes because once they are systematically assembled, they are no longer anecdotes.
Did that make sense? I feel like I rambled a lot. Let me try to illustrate this one other way because I’m not sure that I’m being clear. In my previous example, I started with anecdotes, but we can go in the reverse direction. We can, for example, take a data set from a carefully controlled study, rip one example out of that study, and use it as an anecdote. In other words, once it is removed from the systematically collected body of data it becomes an anecdote.
LikeLiked by 1 person
The truth of a claim is entirely dependent on the source of a claim. When a person *claims* something, they are asserting that they are justified in believing it. When a person says “I claim that it’s raining”, they’re not merely stating a logical proposition. They’re vouching for it and alleging that you are justified in believing it.
In the bizarro world of this artcie, a rational response to “your wife is cheating on you” is “how interesting, you’ve just stated a logical proposition that might or might not be true” not “unless you’re joking, you better have a good reason for saying that or I’m never speaking to you again”.
LikeLike
And to clarify what the genetic fallacy actually is, it’s when you draw conclusions based on the source of a claim in disregard of evidence to the contrary or in a way that forecloses the introduction of contrary evidence. If all you have is the source of the claim, it’s perfectly valid to judge the validity of the source so long as you perform that analysis reasonably, don’t have unjustified confidence in your conclusion, and are open to new evidence.
LikeLike
This is basically just a restatement of my post, so I’m really not sure why you are setting this up as being in disagreement with it.
The only part of this comment that I take issue with is “in disregard of evidence to the contrary or in a way that forecloses the introduction of contrary evidence. ” That is not a requirement for it to be a genetic fallacy. Let’s say, for example, that a scientist writes a paper claiming X. Further, this is the first paper on X, so there is no other evidence at the moment. Now, let’s suppose that someone doesn’t like that author for a totally arbitrary reason. For example, suppose that the scientist is short, and our imaginary person has a bizarre bias against short people. Therefore, the person says, “I don’t accept X because short people are unreliable.” Nevertheless, if a tall person does the same study and finds X, our fictional character will accept that result. In that case, a genetic fallacy is clearly being committed because he is attacking the source not the information (i.e., he is drawing a conclusion based on the source) and his attack is totally irrelevant. Nevertheless, his attack has nothing to do with excluding contrary evidence, and it doesn’t preclude future evidence.
That’s why it is far more accurate to describe the genetic fallacy as, “basing a conclusion on the source of the information when that source is irrelevant to the trustworthiness of the information” (or in the specific case of my post which was dealing with attacks on the source, “attacking a source when that attack is irrelevant to the trustworthiness of the information”).
LikeLike
The problem is solely that the reason is arbitrary and unsubstantiated. Not that it’s about the source of the information rather than the information. It’s perfectly reasonable to believe in Australia but not Atlantis because Australia is described in reputable textbooks and Atlantis is only found in books considered untrustworthy or describing myths.
For example, you say, “As it’s name suggests, the genetic fallacy results from attacking the source or origin of information, rather than the information itself.” This is wrong. It’s perfectly fine to attack the source or origin of the information rather than the information. It’s just a question of whether that attack is reasonable and justified.
LikeLike
“It’s perfectly fine to attack the source or origin of the information rather than the information. It’s just a question of whether that attack is reasonable and justified.” I agree. In fact, that was what this entire post was about. I very clearly explained that it is ok to attack the source if that attack is actually relevant to the trustworthiness of the information.
If read down just a tiny bit further from the section of the post that you quote, you will find these sentences, “In other words, there is nothing wrong with attacking a source, if the source of the information is actually germane to whether or not you should trust the information. So, if someone cites questionable sources like Youtube videos or personal anecdotes, there is nothing wrong with you saying that we shouldn’t trust that information, because the sources actually are unreliable.”
LikeLike
“The truth of a claim is entirely dependent on the source of a claim.” What? That makes no sense whatsoever.
Consider the following claim, “the earth is a spheroid.” The truth of that claim is completely independent of the person making that claim. It doesn’t matter whether it is Ken Ham (an outspoken creationist) or Stephen Hawking or a pathological liar who is saying it. Whether or not the claim itself is true is completely independent of the person saying it. Now, whether or not you should trust the claim is dependent on the source, but the source has no impact on the claim itself.
Regarding your example of a cheating wife, why you think that this post requires the first response? The rational response is, “what is your evidence?” Nevertheless, it’s obvious that the source of the claim would affect how you respond. For example, if the person saying that your wife is cheating is a close and trusted friend who would never make such an accusation unless they were completely certain, then you should probably be concerned that your wife is in fact cheating on you. If, however, your source is a pathological liar or is prone to drawing absurd and unreliable conclusions from limited evidence, then you should be pretty skeptical. Either way, the appropriate response is to ask for their evidence, because even your trustworthy source could be mistaken, and even your unreliable source could be right. This brings me back to my central point in this comment: the source of the claim has absolutely no impact whatsoever on whether or not the claim is true. As the old saying goes, even a broken clock is right twice a day.
LikeLike
You are confusing a claim with the thing claimed. When someone *claims* that the Earth is a spheroid, what they are saying is that they have reason to believe that the Earth is a spheroid that justifies them, and you, believing that the Earth is a spheroid.
It is a silly bit of philosophical navel gazing to claim that if a person says “your wife is cheating on you”, he is saying something different than if he says “I have good reason to think your wife is cheating on you” or “you are justified in believing that your wife is cheating on you”.
LikeLike
So you are just debating the semantics of the word claim? I really have no interest in semantic debates so I won’t pursue that line of argument beyond simply saying that I disagree with you because of sentences like this one, “His claim is that the earth is a spheroid.” I’m not interested in whether or not the person believes what he is saying, I am interested in whether or not what he is saying is actually true (as I think was abundantly clear in the post, which is why I hate it when people waste my time on trivial semantics).
Back to the wife example, why would you assume that no one would ever make that claim without good evidence for doing so? People make claims based on poor evidence all of the time. That’s how rumors get spread. So the fact that someone says, “your wife is cheating on you” is in no way a guarantee that the information is evidence-based, which is why the rational response is to ask for the evidence. If the evidence is, “I saw her doing another guy” then you should probably trust it, but if the evidence is, “my friend said that her friend knows a girl whose cousin knows a guy who says he slept with your wife” then you probably shouldn’t jump to conclusions because things tend to get lost as they get repeated (again, rumors get spread without justification all of the time). Or, once again, your friend may be very untrustworthy. I don’t know about you, but I certainly know lots of people who are prone to jumping to conclusions without good evidence. So if one of them made a claim like, “your wife is cheating” I would be extremely skeptical until I had seen their evidence.
In short, your argument seems to be predicated on the notion that people never make claims unless they have good evidence to back them up, and I completely reject that premise and find it to be laughably absurd.
LikeLike
The point is that although you cannot discredit a result based on its source you may discredit a source based on its results. If a method of inquiry is fundamentally broken then its results are unreliable although incidentally sometimes true (although they produce incorrect causal arguments). You have ranked methods of inquiry by how broken they are and I contend you have done something meaningful which does not implicate the genetic fallacy. Good work.
LikeLike