The image that took me two months to obtain: A window into what it is like to do research

Posted on January 30, 2017 by Fallacy Man

Most people don’t really understand what scientists actually do or what is required to do scientific research. As a result, they don’t appreciate the amount of training and work that goes into being a scientists and conducting research. I personally think that is a problem, because it leads to all sorts of confusion and misconceptions. So I want to try to clear some of that up by telling you all a bit about what I have been working on for the last few months. I hope that this example will help you to understand and appreciate just how much goes into scientific research, as well as clearing up a lot of misnomers. Many people seem to think that scientists are lazy and constantly take short cuts, falsify data, etc., but I hope that this will help you to see why that is rarely the case. I should clarify, however, that because I am going to talk about some of my own work here, there may be a few places where it sounds like I am boasting, but I assure that that isn’t the case. Quite the opposite, in fact. What I want to impress upon you is that experiences like mine are the norm in science, not the exception. I’m a 27-year-old who still carries a valid student ID. I am by no means a prodigy. At best, I’m average.

Two months to get one image

Before I can explain why it took my two months to get one image, I need to give you a bit of background about my current project. I won’t go into the details, but briefly, I am doing a microbiome project looking at how microbiomes (i.e., the bacteria, fungi, etc. that live on an organism) affect disease ecology. So I collected a bunch of samples (swabs in this case), and now I need to extract the microbial DNA from them. Once I have that DNA, I can sequence it and use those sequences to identify the species of micro-organisms that were living on my study subjects as well at looking at relative abundances.

That DNA extraction step has, however, proved quite challenging. You see, I am trying to get DNA from a range of organisms (gram negative bacteria, gram positive bacteria, and fungi), which is challenging because of differences between the groups. Gram positive bacteria, for example, have thick peptidoglycan cell walls that often require a mixture of chemicals to break them apart. In contrast, fungi have chitin cell walls that often require mechanical methods for breaking them apart (e.g., you add some tiny beads to the sample and use a “bead beater” to rapidly bash them against the sample, thus breaking apart the cells). However, not all of these methods work well together, and some methods that work really well for one group actually reduce the DNA yields from other groups. Further, to make things even worse for me, my swabs are very low yield samples, which means that there isn’t much DNA there to begin with.

This is a gel from an extraction method that completely failed. The two bright bands are control samples to ensure that the PCR and Gel worked correctly. If any of the other samples had worked, they would have shown up as similar bright bands. Thus, since my controls worked, but my samples did not, I concluded that this method does not work well for my samples.

This is a gel from an extraction method that completely failed. The two bright bands are control samples to ensure that the PCR and gel worked correctly. If any of the other samples had worked, they would have shown up as similar bright bands. Thus, since my controls worked, but my samples did not, I concluded that this method does not work well for my samples.

The result of all of this is simply that I have spent two months working in the lab trying to get an extraction method that will work reliably and consistently for all of the groups that I am interested in, even when they are very unabundant. Let me explain a bit of what that has looked like. First, I had to make a whole bunch of mock samples with known quantities of the organisms that I was interested in, that way I could test my extraction techniques on them to see if they worked. Then, I picked the method that I thought was the most promising, and I tried it on a bunch of test samples. That took a full day because there were several long steps where I had to let the samples incubate, precipitate, etc. The next day, I used a PCR (polymerase chain reaction) to try to amplify the DNA from those samples. This basically just replicates the regions of the DNA that I am interested in, that way there are enough copies for me to work with. Finally, I took those PCR products and ran them on a check gel. I won’t go into the details of what that means, but it is just a simple method for seeing wither or not you got DNA amplification, how much you have, and how long the DNA fragments are. So on a gel, if I was successful and my methods worked, I should get nice bright bands of DNA. However, I usually got gels that look like the one to the right. The only bands there are the positive controls and the ladder (you always run positives and negatives in a PCR so that you know if the it is the PCR or the extraction that failed, and the ladder is a standard that lets you tell how large the DNA fragments are).

Following that first failure, I did a bunch more reading on the method I was using, talked to several colleagues, then made some modifications to the protocol and tried again, but once again, I was met with failure. By this point, I was already four days in, but things were just getting started, because this cycle continued to repeat itself over and over again. I would try a method, it would fail, I would modify the method by changing the chemicals, temperatures, times, etc., that would fail, I would modify it again (or try an entirely different method), that would fail, so on and so forth.

This is a gel from a method that actually worked. This time, you will notice that nearly every well has a band, indicating that the extraction worked (the cells without bands were negative controls). You will also notice that some of these are smeared. That is because I was testing several methods, and I will not be using the method that produced the smears.

This is a gel from a method that actually worked. This time, you will notice that nearly every well has a band, indicating that the extraction worked (the cells without bands were negative controls). You will also notice that some of these are smeared. That is because I was testing several methods, and I will not be using the method that produced the smears.

This lasted for two months, during which, I worked in the lab at least six days a week, often working over ten hours a day trying to find a reliable extraction technique, and I want to stress that word “reliable” because it’s important. There were multiple times when I found a method that “worked” in that I could get DNA from many samples, but those methods did not give me good, consistent coverage over all of the groups of organisms that I was interested in, especially when low quantities were present. In other words, I could have done my study using one of those methods, and I would have gotten what appeared to be good results that I could easily have published, but the results would have been misleading because I didn’t use a reliable method that could consistently detect all of the groups I was interested in. Therefore, I kept trying over and over again, until finally, last week, I got the image that you see on your left. This time, you will notice that there are DNA bands, and although I have not labelled the image, I got bands even from samples that only had a handful of cells. I have also replicated these results, and they are consistent across my organisms of interest. So I finally have a reliable method, but it took me two months to get here.

Note: there are actually some issues with that gel such as smears and double bands, but that is just because I was running several methods at once. So the lanes that look messed up are from the method that I will not be using, and the lanes that look good are from the method that I will be using. There are also still a few tweaks (particularly to the PCR procedure) that I will make to clean things up a bit more before extracting the DNA from my actual samples.

The big picture: what it takes to do research

Now that you understand my suffering for the past two months, I want to talk a bit more generally about what has been required for this project. First, at the very outset I had to read a tremendous amount of scientific literature. I needed to understand the topic that I was going to study, I needed to know what other people had already found, and I needed to identify the current gaps in our knowledge. This type of background work is absolutely essential in science, and it can be quite time consuming. In many cases, this requires reading hundreds of papers.

Following that, I had to actually design a study, which means that I needed to know enough about experimental design and statistics to design a project that would let me reliably answer the questions that I was interested. So once again, a lot of background knowledge and prior training was necessary.

Simply designing a project clearly isn’t enough, however. I actually needed to do it, and for that I needed money. So, I spent several weeks writing grants to lots of different funding agencies to try to get money for this project. Several of them rejected me, but several others accepted my proposal so I got the funding. During that same time period, I also applied for research permits and filled out ethics applications so that I could actually do the project. Fortunately, all of that came together, so once I had money, permits, and ethical clearance, I immediately moved on to the next phase and actually went out into the field and collected my samples (that was also quite a challenge because it required hiking up ridiculous rainforest streams, climbing cliffs and waterfalls, etc.).

After getting the samples, I started on the lab work that I have been talking about thus far, and now that I have a working protocol, I can move on to the next stage: actually getting DNA from my real samples. Once I have the DNA from those samples, I will amplify and sequence them, and that will finally give me the actual data which I will then do a series of statistical analyses on. Following those analyses, I will write a paper on the results and submit to a journal (it will probably be another 4–6 months before I have something submitted, maybe longer).

If we put all of that together, you should start to be able to see just how much effort and how many skills are required for a project like this. I needed to have an enormous amount of background knowledge on the topic and on experimental design just to plan the study, I needed writing skills to get the grants/permits/ethics clearance, I need field skills to get the samples, I needed lab skills and a knowledge of chemistry to get the extractions working, once I finish the lab work I will need statistical skills to analyze the data, and I will need writing skills again to actually craft a paper. Perhaps most importantly though, I really need to understand the ecology of my species in order to make sense of the data that I am collecting. In other words, it’s not just enough to know a bit about the background and get the raw data, I need to actually understand things well enough that I can make sense of my results and see how they fit into the big picture. So a tremendous amount of work goes into a project like this, and a lot of knowledge and skill is required.

At this point, I again want to clarify that I’m not trying to convince you that I’m a brilliant, naturally gifted researcher, because I’m not. What I am, however, is highly trained. This brings me to perhaps my most important point in this post: science is hard, and it takes lots of training and work. I couldn’t have done a project like this when I started my scientific training almost a decade ago, and, honestly, I couldn’t even have done it a few years ago. It takes years of extremely hard work to become an academic because there is so much that you need to know. There is an overwhelming amount of knowledge that needs to be acquired and skills that need to be mastered in order to do research like this, and it takes a long time to slowly build up that knowledge and those skills. As a result, I get very upset when people who have never taken an advanced science course, never set foot in a lab, never learned statistics, etc. pretend to know more than scientists and claim that scientists are wrong on vaccines, climate change, GMOs, evolution, etc. To be clear, I’m not being elitist here. I’m not suggesting that scientists are better or smarter than everyone else, but they are highly trained and experienced, and that matters. The idea that you will understand a topic better than scientists simply by reading some websites and books is insane and extremely arrogant. Scientists are carefully trained on how to design experiments and analyze data, they spend an inordinate amount of time reading every available paper on the topics that they study, and they spend years actually doing research. That type of training, knowledge, and experience isn’t going to be toppled by a few Youtube videos.

Note: I want to be clear that I am not trying to be dismissive of citizen science projects, because I actually think that those are fantastic, largely because they help to train the public to think like scientists. Similarly, I am not trying to disparage anyone who is trying to learn about science. As I said, being a scientist is as much about training and diligence as anything else, and anyone can learn how to be a scientist. My point is simply that you shouldn’t place too much weight on your degree for Google University.

It’s a group effort

I’ve been writing this post in the first person, but I want to be totally clear that I have not been doing this alone. I have had a tremendous amount of help and support at every step of this. I have gotten help and advice from my fellow graduate students who are in the lab working on their own projects, I have gotten advice from post-doctoral researchers who recently finished their PhD’s, and I have gotten advice and help from my advisers, each of whom is an experienced researcher who spends much of their time passing on their knowledge to students. Further, this is the norm in science and has been the norm for me at every level of my training. I have had great mentors from day one and have learned a tremendous amount from working with them. Once again, that’s a type of practical learning that just can’t be matched by surfing the web. Also, you should realize that science is extremely collaborative, and most projects get input and advice from a whole host of people who don’t make it onto the actual author list. This is, in my opinion, one of the greatest strengths of science, because everyone has a different set of skills and a different knowledge base, so when those skills and knowledge bases are combined, you end up with an extraordinary and diverse amount of knowledge and ability being funneled into one project.

Most scientists aren’t incompetent or unethical

The final thing that I want to deal with in this post is the notion that scientists are either hopelessly incompetent or horribly unethical. I hear these claims all of the time, and they are ludicrous. For example, when it comes to climate change, I frequently hear people say things like, “the answer is obvious, it’s the sun, but scientists just plow forward with their beliefs instead of looking at the obvious answer” or “but the climate has changed repeatedly in the past, so it’s silly to think that the current change isn’t natural.” Claims like that are absurd. Scientists aren’t incompetent, and they have, in fact, carefully tested the natural drivers of climate change (including the sun) and consistently found that natural causes of climate change cannot explain the current warming, but our greenhouses gasses can (more details and sources here). Scientists spend a great deal of time testing every reasonable possibility that they can think of, as well as systematically refining their methods and analyses to ensure that they get accurate results. That is one of the things that I hope you will take away from my two months of trying to get a reliable extraction method working. There were lots of points that I could have stopped and used a method that I knew wasn’t accurate or reliable, but I didn’t do that because I wanted to make sure that my final results will be correct.

Once again, this type of situation is the norm for scientists. Talk to any researcher, and they will tell you very similar stories of weeks, months, or even years spent trying to get a method to work reliably, trying to refine an analysis to make it more accurate, etc. We all spend an inordinate amount of time trying to make sure that we are doing things correctly and are accounting for all reasonable possibilities. To be clear, scientists aren’t perfect, and mistakes certainly get made, but most of us care tremendously about producing high quality research because we are curious people who want answers. That’s why we went into science (it certainly wasn’t for the money, because there’s not much of that). I chose and designed my current project because it is a topic that I care about. I want my results to be correct and accurate because I want to know what the answer is. That type of curiosity is a huge driving force in science, and it forces us to be diligent.

This brings me, finally, to the topic of wide-spread corruption. Many people are under the delusion that scientists are just in it for money, and we go around falsifying data whenever it disagrees with us. That idea is, however, total nonsense. If I didn’t actually care about the results, if I was willing to falsify data, then why on earth would I spend two months working 60+ hours a week trying to get a reliable extraction method working? If I was just going to make up the results, then I could have just gone ahead with one of the crappy, unreliable methods. For that matter, I could have just skipped the lab work altogether and sat down at my compute and generated fake results. Similarly, the idea that I would get to the end of this project only to have someone pay me to keep the results quiet or to change the results is nuts. Look, by the time that it is all said and done, I will have invested close to a year of my life into this project. I have poured everything into it, and there is no way on earth that I am going to get to the end and either not publish or change my results just because someone offers me a few thousand dollars. I am not so provincial and neither are the majority of scientists. Again, we don’t do this for the money. If money was what we cared about, we would have chosen a different field. We do this because we want to know how things work. We do this because we care about expanding mankind’s knowledge and making the world a better place, and the vast majority of us would not be willing to bury research for a simple bribe. To be clear, there are bad apples in every group, but again, what I am describing here is not out of the ordinary. Go talk to any scientists anywhere in the world and they will have similar stories, because this is the norm for science.

Posted in Nature of Science | Tagged , | 3 Comments

How long is long enough? Do we need more climate data?

Posted on January 23, 2017 by Fallacy Man

climate change data global warming warmest years on record2016 was the hottest year on record, making it the third year in a row to set a new temperature record. Indeed, the past few decades have been so warm, that anyone who is 31 years old or younger has never experienced a single month when the earth’s temperature was below average (that’s using the 1951–1980 average that is generally used for benchmarking global temperatures; NASA data). That’s pretty striking evidence that the planet is warming. Nevertheless, there are many who object to this and claim that we simply don’t have enough data to know what is really going on. They point to the fact that our global records only go back to 1880 and insist that 130+ years is nothing when you consider the entire history of planet earth. However, this argument ignores several key points that I will elaborate on. First, we actually do know a lot about the climate prior to 1880, and we have used those data to understand how the climate works. Second, by using our understanding of past climate, we have been able to test the current drivers of climate change and determine with an extremely high degree of confidence that the current warming is not natural.

How long is long enough?

Before I deal with the actual evidence itself, I want to point out a fundamental problem with this argument. If you think that 130+ years isn’t enough data, then how much will be enough? How long does the trend need to go to convince you? Think about it, if the argument is that 130+ years is nothing compared to the billions of years that earth has been around for, then there is probably no realistic amount of data that would convince you. For example, would 200 years convince you? I doubt it, because in the grand scheme of things, those extra 70 years are also minuscule. To put this another way, people have been saying “there isn’t enough data” for decades now, even as more data has continued to accumulate. Back in 2000, for example, tons of people said that we don’t have enough data and the trend probably won’t continue, but 16 additional unusual (and often record setting) years of data have been collected since then, and many of them still aren’t convinced. So how long does the trend have to last for before you’ll accept it?

My point is that this argument sets up a non-falsifiable situation that allows people to appear rational by demanding more evidence, when the reality is that no amount of evidence will convince them. To put that another way, people who use this argument are picking an arbitrary amount of data that is necessary, rather than actually looking at the quality and content of the data that we already have. This argument is really a cop-out. It is an excuse for blind ignorance, rather than a logical argument. So if you are prone to using this argument, I hope that you will set it aside for a minute and actually look at the data that we have gathered.

Past climate data

When we talk about global temperature records, we are generally referring to the period starting with 1880, because that is when we had sufficient, real-time measurements from around the world to confidently state the global temperature. However, we do in fact have climate data going back much, much further thanks to things like ice cores, the ratios of certain chemicals, etc. So although we may not be able to confidently state a precise global mean temperature before 1880 that is directly comparable to our modern measurements, we still have a really good understanding of what the climate was like. For example, Martin et al. (2005) used the Mg/Ca ratios of benthic foraminifera (a marine protist) to trace ocean temperatures back for 90 thousand years (which is substantially longer than 130+).

This is really important, because we can use those past climate data to understand things like how solar cycles and CO2 affect the climate. Scientists have, of course, done this, and here is what they found in a nutshell. The earth goes through prolonged cycles known as Milankovitch cycles that affect the earth’s eccentricity, precession, and obliquity (i.e., orbit, tilt, and axis) and this affects the amount of energy entering the earth or the location/season at which it enters. This (as well as other factors like solar activity) often causes a small amount of warming (often regional), and that warming causes the oceans to release the CO2 that is stored in them (Martin et al. 2005; Toggweiler et al. 2006; Schmittner and Galbraith 2008; Skinner et al. 2010). That CO2 then drives the bulk of the warming (this is why there is a brief “lag” in the climate data; Lorius et al. 1990 ; Shakun et al. 2012). So in short, past events of global warming occurred whenever something caused enough small-scale warming (often regional) for the oceans to release CO2, and that spike in CO2 caused most of the actual warming (more details here).

Our current warming

Now that we have established the drivers of past climate change, we can look at our current situation to see whether or not natural factors are causing the current warming (spoiler alert, they aren’t). You see, the thing that the “only 130+ years of data” argument ignores is that we already understand causal relationships when it comes to the drivers of climate change. In other words, scientists aren’t just sitting back and saying “well it’s been warming for 130+ years, therefore it must be us and it will keep on warming until we change.” Rather, they have carefully examined the cause of the warming, and it is the understanding of that cause that allows us to be so confident.

So what is the cause? You probably guessed it; it’s CO2. The image to your right shows the earth’s atmospheric CO2 concentration over the past 800,000 years. You’ll notice that it is substantially higher now than it has been during any point in that time period (surely 800,000 years is long-term enough). Also, remember that we know from our studies of past climate change that CO2 is a major factor in driving planetary warming. At this point, the situation is really quite simple: we know that CO2 traps heat and is important for regulating our climate, we know that CO2 has been a major driver of past climate change, and we know that we have greatly increased the CO2 in our atmosphere, therefore, the only logical conclusion is that we are causing the climate to change. Please note that this conclusion is not based on 130+ years of warming. Rather, it is based on understanding physics and the factors that drive climate change. Indeed, scientists predicted that burning fossil fuels could change the climate way back in 1896. So the recent warming is simply the confirmation what scientists predicted over 100 years ago.

To put this another way, we can demonstrate that we are the cause using a simple syllogism.

  1. In the past, large increases in CO2 have caused the planet to warm
  2. We have caused a large increase in CO2
  3. Therefore, we are causing the climate to warm
This figure from Hansen et al. 2005 shows the effect of both the natural and anthropogenic drivers of climate change. Notice how only anthropogenic sources show a large warming trend. Also, see figure 2 of Meehl et al. 2004.

This figure from Hansen et al. 2005 shows the effect of both the natural and anthropogenic drivers of climate change. Notice how only anthropogenic sources show a large warming trend. Also, see figure 2 of Meehl et al. 2004.

Nevertheless, you may protest and say, “what about volcanoes, the sun, etc.” Well, first off, volcanoes actually only emit a very tiny amount of CO2 (less than 1% of what we produce Gerlach 2011). Second, scientists have carefully looked at volcanoes, the sun, and other natural drivers of climate change, and they simply cannot explain the current warming without including our greenhouse gas emissions (Stott et al. 2001; Meehl et al. 2004; Hansen et al. 2005; Allen et al. 2006; Lean and Rind 2008; Imbers et al. 2014).  In other words, we know that the current warming isn’t natural, because we have tested the natural causes of climate change, and they all come up short (even when combined together; more details here). Output from the sun, for example, simply doesn’t correlate with our current warming, therefore we know that it isn’t the sun that is driving it (note: the actual tests are far more complicated than simple correlations, because they both control for confounding variables and combine the effects of variables, but it is often convenient to think about it as simple correlation to start with, then build complexity on top of that). To be clear, these tests have also shown that the current warming is not 100% human induced. There is some natural background fluctuation, but the bulk of it is man-made, and it will continue if we don’t reduce our fossil fuel use.

It is probably also worth mentioning that we are certain that the increase in CO2 is from us because of isotope ratios. I explained this in more detail here, but in short, carbon has two stable isotopes (C13 and C12), and the ratio of those two isotopes in fossil fuels is different from the ratio that is in the atmosphere. Therefore, if the CO2 is from us burning fossil fuels, we would expect that atmospheric ratio to shift to be more like our fossil fuels, which is exactly what has happened (Bohm et al. 2002; Ghosh and Brand 2003;Wei et al. 2009). These ratios are like fingerprints that demonstrate beyond any reasonable doubt that the CO2 is from us.

Confirmation via models

At this point, you may be tempted to think that all of this is well and good, but we can’t really know what is happening without more data. I disagree, but there are still additional pieces of evidence that we can bring to bear. For example, we can be highly confident that our current understanding is correct thanks to models (as well as other data that I’ll talk about later). Scientists are a pretty clever bunch, and what they realized is that we can construct models based on our current understanding of how climate works, and use those models to make predictions about the future. If our understanding is correct, then the models should make accurate estimates, and if our understanding is wrong, the models should make incorrect estimates (science thrives on testable predictions like this). Granted, there is a certain amount of uncertainty around this, because the models are reliant on the skills of their programmers, as well as natural factors that are out of our control (e.g., if the output of the sun changes and is different from the model input, that will cause the model to be wrong, even if our basic understanding of the climate is correct). Nevertheless, these models have actually done a remarkably good job of predicting the current warming (Hansen et al. 2006; Frame and Stone 2012; Rahmstorf et al. 2012; Cowtan et al. 2015; Marotzke and Firster 2015). Now, yes, I know, you have read on the internet that the models have all been wrong. Well, I hate to break it to you, but not everything on the internet is true. The claims that the models have been wrong are based on cherry-picking and misrepresenting how the models work. The actual scientific analyses have found that they have done a good job of predicting the current warming (more details here and here). Also, it is worth mentioning that the models have been refined over time because our understanding of the climate has improved over the years. However, those refinements simply improve the accuracy of the models, they don’t change the overarching trend or predictions.

Satellite measurements

Finally, I want to draw your attention to one final piece of evidence which, in my opinion, is the single most conclusive demonstration that we are causing the planet to warm. When energy from the sun hits the earth’s surface, much of it is radiated back of as long-wavelength, low-energy infrared radiation (IR, aka heat energy), and CO2 in our atmosphere is capable of absorbing those long wavelengths. Thus, it traps some of that heat energy before it leaves the earth. This is a scientific fact that no one disputes, and it is actually a good thing, because the earth would be inhospitably cold without CO2 and the other greenhouse gases. Nevertheless, it can be a problem, because if that CO2 increases, then more heat is trapped and the earth warms. Again, we know that this is largely what drove past climate changes.

Now that we have those facts in place, we can make another testable prediction. If our CO2 is causing the planet to warm, then the amount of energy entering the earth should be roughly constant, but the amount of energy that is leaving the earth should decrease significantly because some of it is being trapped by our CO2. Scientists have actually measured this using satellites, and just as expected, the energy from the sun is basically unchanged, but less energy is leaving at the specific wavelength that CO2 traps (Harries et al. 2001; Griggs and Harries. 2007; Huang et al. 2010). This is extremely clear evidence that we are causing the planet to warm, and if you want to say that the warming isn’t from us, then I want to know where that IR is going. We know that the roughly same amount is coming in, but less is going out, so if it’s not our CO2 that is trapping it, what is it? (note: recently, the sun’s output has decreased slightly, but it is not enough to explain the decrease in IR, and the pattern hasn’t been consistent; i.e., the sun’s output does not correlate well with the decrease in IR, but our CO2 does).

In conclusion, let’s put all of the pieces together. We know that CO2 traps heat and warms the planet, we know that increases in CO2 caused past climate change, we know that we have greatly increased the CO2 in the atmosphere, and we know that CO2 is currently causing the planet to trap excess heat. The conclusion from that is inescapable: we are causing the planet to warm. We do not need more data to be confident in that conclusion.

Note: To be clear, I’m not suggesting that we should stop studying the climate, we certainly should and there is still much to learn, but the evidence that we have so far is sufficient to be highly confident that we are causing the climate to change. To use an analogy, there is still a lot to learn about the human body and we should keep studying it, but we have plenty of data for me to confidently state that your liver isn’t the organ that is responsible for pumping blood.

Literature Cited

  • Allen et al. 2006. Quantifying anthropogenic influence on recent near-surface temperature change. Surveys in Geophysics 27:491–544.
  • Bohm et al. 2002. Evidence for preindustrial variations in the marine surface water carbonate system from coralline sponges. Geochemistry, Geophysics, Geosystems 3:1–13.
  • Cowtan et al. 2015. Robust comparison of climate models with observations using blended land air and ocean sea surface temperatures. Geophysical Research Letters 42:6526–6534.
  • Frame and Stone 2012. Assessment of the first consensus prediction on climate change. Nature Climate Change 3:357–359.
  • Gerlach 2011. Volcanic versus anthropogenic carbon dioxide. EOS 92:201–202.
  • Ghosh and Brand. 2003. Stable isotope ratio mass spectrometry in global climate change research. International Journal of Mass Spectrometry 228:1–33.
  • Griggs and Harries. 2007. Comparison of spectrally resolved outgoing longwave radiation over the tropical Pacific between 1970 and 2003 Using IRIS, IMG, and AIRS. Journal of Climate 20:3982-4001.
  • Hansen et al. 2005. Earth’s energy imbalance: confirmation and implications. 308:1431–1435.
  • Hansen et al. 2006. Global temperature change. Proceedings of the National Academy of Sciences 10314288–14293.
  • Huang et al. 2010. Separation of longwave climate feedbacks from spectral observations. Journal of Geophysical Research 115:D07104
  • Imbers et al. 2014. Sensitivity of climate change detection and attribution to the characterization of internal climate variability. Journal of Climate 27:3477–3491.
  • Lean and Rind. 2008. How natural and anthropogenic influences alter global and regional surface temperatures: 1889 to 2006. Geophysical Research Letters 35:L18701.
  • Lorius et al. 1990. The ice-core record: climate sensitivity and future greenhouse warming. Nature 139–145.
  • Marotzke and Firster 2015. Forcing, feedback and internal variability in global temperature trends. Nature 517:565–570.
  • Martin et al. 2005. Role of deep sea temperature in the carbon cycle during the last glacial. Paleoceanography 20:PA2015.
  • Meehl, et al. 2004. Combinations of natural and anthropogenic forcings in the twentieth-century climate. Journal of Climate 17:3721–3727.
  • Rahmstorf et al. 2012. Comparing climate projections to observations up to 2011. Environmental Research Letters 7:044035.
  • Schmittner and Galbraith 2008. Glacial greenhouse-gas fluctuations controlled by ocean circulation changes. Nature 456:373–376.
  • Shakun et al. 2012. Global warming preceded by increasing carbon dioxide concentrations during the last deglaciation. Nature 484:49–54.
  • Skinner et al. 2010. Ventilation of the deep Southern Ocean and deglacial CO2 rise. Science 328:1147-1151.
  • Stott et al. 2001. Attribution of twentieth century temperature change to natural and anthropogenic causes. Climate Dynamics17:1–21.
  • Toggweiler et al. 2006. Mid-latitude westerlies, atmospheric CO2, and climate change during the ice ages. Paleoceanography 21:PA2005.
  • Wei et al. 2009. Evidence for ocean acidification in the Great Barrier Reef of Australia. Geochimica et Cosmochimica Acta 73:2332–2346.
Posted in Global Warming | Tagged , , | 6 Comments

You’re probably wrong

Posted on January 16, 2017 by Fallacy Man

I want to begin this post with a simple thought experiment. It is not original with me, and unfortunately, I don’t recall where I first heard it, but I think that it is a useful exercise. I want you to think about all of the thousands of different views, beliefs, ideas, etc. that you hold. Think about all of the things that you think are true. Now, estimate the percentage of them that are actually true. In other words, if you are honest with yourself, you’ll have to admit that you are almost certainly wrong about at least some of your views. It’s extremely unlikely that you are right 100% of the time. So how often do you think that you are right? 80% of the time? 90% of the time? 95%? Are you presumptuous enough to go to 99%? Regardless of what number you settled on, it was surely something less than 100% (unless you are hopelessly arrogant). You’re not correct 100% of the time, and neither am I, and neither was Stephen Hawking, and neither was Einstein, etc.

Now, having established that you are wrong about at least some of your views, I want you to divide your views into things you’re right about and things you’re wrong about. Go ahead and try it. You can’t do it, and neither can I. You inherently don’t know which of your views are wrong, because if you knew that a view was false, then you wouldn’t hold it as a true belief (as least I hope that is the case). In other words, we all hold the views that we hold because we think that they are true. So we inherently think that all of our views are right even though, if we are honest, we have to admit that at least some of them are almost certainly false.

At this point, we have established two fundamental and critically important points that should act as guiding concepts for you.

  1. Some of your views are wrong
  2. You don’t know which ones are wrong

The implication of those two facts is obvious and inescapable: you must always be willing to consider the possibility that you are wrong about any given topic. This is one of the guiding principle of skepticism, and it is one that everyone should take to heart. It does not matter who you are, it doesn’t matter how smart you are, and it doesn’t matter how well-educated you are, you could still be wrong about any of your views. Therefore, you must always consider contrary evidence when you are presented with it (see note at the bottom).

In concept, this sounds simple enough, but actually practicing it is admittedly difficult. We like to think that we are right. We take comfort in thinking that we understand the way that the world works, and being faced with evidence that disagrees with our views is unpleasant. We don’t like being wrong. As a result, we tend to cling to any source that agrees with us and blindly reject any source that disagrees with us. More often than not, when we are presented with a source or argument that disagrees with one of our views, we simply write it off as “fake news,” we claim that the author had an agenda, we invent elaborate conspiracy theories, we cherry-pick evidence to hurl back at it, etc. We are willing to do anything that it takes to avoid facing the harsh truth that we might be wrong. This is called motivated reasoning, and it is a very serious problem because it prevents you from actually considering evidence and making rational decisions. Fortunately, it is a problem that you can overcome.

You can train yourself to be skeptical. The first step in that is simply to admit that you might be wrong, and, following that, force yourself to actually consider contrary evidence and carefully fact check everything. When you read an article that you like, don’t instantly assume that it is true and hit the “share” button. Rather, stop and think carefully about it. Look at the arguments that it is making, look at the sources that it is citing, fact check the article, read the opposing articles, talk to people who hold the opposing view, etc. Similarly, when you see an article that you disagree with, don’t automatically assume that it is false. Don’t assume that it is “propaganda” or “fake news.” Again, fact check it. There are lots of great fact checking websites out there, fact checking fake newsand you should use them. Try as hard as you can to find information that supports the opposing position (from good sources, of course). In other words, you should try to disprove your own position.

What most people do is simply look for information that supports their view, rather than looking for information that opposes it, and that’s a serious problem. Indeed, it leads to what is known as a confirmation bias. Most people “fact check” by looking for other articles that agree with them. They, of course, easily find them; then they instantly assume that those articles are true, and the existence of those articles further bolsters their view. Do you see why that is a problem? It is a feedback loop that will always leave you with the same view that you started with. There will always be other people who think you’re right, so if you only look at what they have to say, you will only ever see arguments that agree with you, and you will never have to actually consider the possibility that you are wrong. That is a monumental problem.

On that note, try to avoid echo chambers. Don’t unfriend everyone who disagrees with you, and don’t block sources that say you’re wrong. Rather, you should engage with those people and sources. See what they have to say, listen to their arguments, and always be willing to be wrong. I don’t want to get political here, but politics present a very good example. In recent months, I have seen a large number of people reach the conclusion that only one major news company is trustworthy, and all of the others are agenda driven liars. Thus, they now get their news from only one source. That would be problematic regardless of which source they chose, but it is especially problematic since their chosen source (which won’t be named) is notoriously biased towards one extreme of the political spectrum. Thus, these people have created an echo chamber for themselves, where all of their “news” comes from people who agree with them, and any source that disagrees with them is automatically considered to be false. That is a huge problem regardless of which end of the political spectrum it falls along, and you should strive to avoid it. Get your information from multiple sources, fact check, and actually read/watch and consider opposing arguments. Also, make sure that the sources that you are using are actually legitimate sources. Although you shouldn’t automatically assume that a source is fake news just because it disagrees with you, there are, nevertheless, a lot of fake news sources out there, and you should be careful to avoid them. Similarly, be leery of sources that are known to have an agenda, because they tend to be biased.

To be 100% clear here, in this process, you must use good sources and actual evidence. People often abuse this concept of being “open minded” and use it to justify believing in unsubstantiated nonsense. Being willing to be wrong, and being willing to accept a position despite a lack of evidence for it are two very different things.

This applies to everyone, regardless of what views you hold and what degrees and qualifications you posses.

This applies to everyone, regardless of what views you hold and what degrees and qualifications you posses.

In short, you should always consider the possibility that you are wrong, carefully fact check everything (especially if it agrees with you), listen to opposing arguments, avoid echo chambers, get information from multiple sources, and even try to discredit your own views rather than trying to support them. I realize that this isn’t easy to do. It is against our nature to be that willing to dismiss our views, but in our modern age of misinformation, this type of skepticism is invaluable and you should strive for it in all areas of life (science, politics, religion, ethics, etc.). I doubt that anyone has ever achieved perfect skepticism (I certainly haven’t), and we are all prone to confirmation biases (myself included), but this is something that you can get better at. You can train yourself to be a skeptic, and as with most things, the more that you do it, the easier that it will become. Make fact checking a habit, rather than something that you only do when you want to disprove someone, and above all else, be willing to be wrong.

Note: Some people may challenge my thought experiment by pointing out that not being able to know which of your views is false is not the same thing as not being able to know that some of your views are true. In other words, you could argue that it is possible to divide your views into things that you are 100% certain are true and things that you are less than 100% certain about. I agree that you can be more certain about some things than others, but I disagree that 100% certainty is ever possible. There are lots of philosophical reasons for this, some of which I discussed here (for example, look up Descartes, the brain in a vat argument, etc.). More practically, however, I think that claiming 100% certainty is the epitome of arrogance. By claiming that you are 100% certain about something, you are saying that you are so knowledgeable, intelligent, and infallible that it is impossible for you to be wrong on a given topic, and that is a rather extreme claim to make. I don’t care who you are, you aren’t perfect enough to achieve 100% certainty. It is always possible that some piece of evidence has alluded you or some cognitive bias has crept in. Thus, you should always be willing to be wrong. To be clear, I’m not suggesting that you can’t be highly certain about some things. You can reach strong conclusions based on the available evidence, but you should always be willing to challenge those conclusions when presented with new evidence (to be clear, I mean actual, proper scientific evidence, not anecdotes, assumptions, conjecture, conspiracy theories, etc.).

Note 2: As several people have pointed out in comments, mathematics are a legitimate exception to my statements regarding absolute certainty (as are the rules of logic) because they are abstract and do not rely on observations of the physical universe. In other words, 2+2 must always equal 4 even if we are living in the matrix, being deceived by an evil genius, etc. Nevertheless, that is clearly quite different from the types of views and beliefs that I was addressing in the post.

Related posts

Posted in Uncategorized | Tagged | 10 Comments

Debunking “The doctors case for homeopathy” by WDDTY: A case study in critically evaluating internet articles

Posted on January 9, 2017 by Fallacy Man

I am very pleased to publish my blog’s first guest post. A reader contacted me a little while ago to see if I would take a look at an essay he had written debunking some of the nonsense that What Doctors Don’t Tell You had written about homeopathy, and although I don’t generally do guest posts, I thought that his piece would be useful to others as an exercise in spotting false information, a rebuke of WDDTY, and a reference for the flaws in many arguments in favor of homeopathy. Therefore, I offered to post it here. Enjoy.

vial of homeopathic medicineI want to explain the problems with an article found in What Doctors Don’t Tell You (WDDTY) titled “The doctors’ case for homeopathy.” This piece is full of misinformation, and it serves as an ideal case study of what I’ve observed to be WDDTY’s typical behavior. Here’s the full list of problems I found with the article, which I will elaborate on below:

  • Cherry picking throughout
  • Making arguments that WDDTY itself admits are invalid
  • Huge over-reliance on anecdotes
  • Outright lies
  • Referencing bad data/bad science
  • Straw man fallacies
  • Deceitful rhetoric, ad hominems, and hyperbole
  • Missing references (that is, unsubstantiated claims)
  • Non sequiturs coupled with hypocrisy
  • Massive internal contradictions
  • Misrepresenting information and quotes from trusted sources

…but all written in a manner that at first appears to be balanced, transparent, and thorough. It is none.

What follows should serve as:

  1. A clear demonstration that the evidence WDDTY provides in favor of homeopathy is unreliable or wrong
  2. A thorough indictment of WDDTY
  3. A cursory lesson in recognising dishonest or fallacious arguments
  4. A cursory lesson in good scientific practice, and good scientific writing

Let’s take a look at each of the problems:

 

Making arguments that WDDTY itself admits are invalid

The broad pattern of the article is:

  1. Make a lengthy argument
  2. Swiftly concede that the argument made is invalid
  3. Move on to another (or, even, repeat the same) bad lengthy argument
  4. Back to step 2.

Most of the article’s content is rendered useless by these two admissions alone:

  • “Such stories can be dismissed as examples of a particularly dramatic placebo effect or even coincidence”
  • “of course, the fact that pharmaceutical medicine has a poor evidence base doesn’t necessarily make a stronger case for homeopathy.”

With these two remarks the writers have dismissed most of their own arguments. But they know full well that readers will ingest the broad points they’ve made, and gloss over the admissions of their worthlessness. In fact, the subtle admission only makes them (falsely) appear so much more reasonable and balanced. Clever.

This article also claims that scientific studies are thoroughly unreliable, before trying to make a case for homeopathy, using… scientific studies! It’s a neat example of the kind of shameless cherry picking that WDDTY repeatedly uses.

 

Overreliance on anecdotes

“GPs who prescribe homeopathy acknowledge that their medicines’ mode of action is difficult to explain scientifically, but they cannot deny the often startling evidence of their own eyes.”

Yes, we can deny the evidence of our own eyes. We have to question such evidence, because we know how susceptible it is to (conscious or unconscious) bias, error and misinterpretation. Anyone who’s read Daniel Kahneman’s “Thinking, Fast and Slow” will know that our own interpretations of our own limited experiences are unreliable. This is why we perform blinded, randomized, controlled trials. The “evidence of our own eyes” regarding homeopathy is perfectly well explained by proven phenomena: the placebo effect, external confounders, and regression to the mean (a phenomenon whose named absence in a lengthy article pretending to be balanced about homeopathy is notable). WDDTY’s assertion here shows just how happy the magazine is to accept completely unreliable information — anecdotes — so long as it supports WDDTY’s position.

Remember, the anecdotes that make up most of this article “can be dismissed as examples of a particularly dramatic placebo effect or even coincidence”. So including them was just dishonest.

“…but they’re not uncommon.”

It’s true that anecdotes are not uncommon. They’re everywhere, and in support of anything you can name. Commonality of a belief doesn’t make it any more true.

 

Outright lies

Sadly, this is where WDDTY comes into its own.

“from the very beginning, there’s been a turf war between two opposing medical systems: the pharmaceutical doctors and the homeopathic doctors.”

There are no such things as “pharmaceutical doctors.” GPs use an extensive array of different treatments (including no treatment); some of those treatments are drugs. (Those same doctors would embrace homeopathic treatments if only the evidence supported them.) The obsessively black-and-white approach that WDDTY takes is insidious and only makes its advocates look simple.

“[In Europe] studies have shown that primary-care practices offering homeopathy have better patient outcomes than pharma-only practices, and often at lower cost.”

To repeat: there are no “pharma-only practices”.

“the monoculture of pharmaceutical drugs”

To repeat: there is no “monoculture of pharmaceutical drugs”.

“In 2010, [the British] Science and Technology Committee report recommended … banning any further homeopathic research”

Here’s what the report actually said:

“We would challenge Professor Harper’s comment that research funding should be directed towards exploring theories that are not scientifically plausible. Research funding is limited and highly competitive. The Government should continue its policy of funding the highest quality applications for important scientific research determined on the basis of peer review.” Then: “We recommend that the Government Chief Scientific Adviser and Professor Harper, Chief Scientist at the DH, get together to see if they can reach an agreed position on the question of whether there is any merit in research funding being directed towards the claimed modes of action of homeopathy.” Also: “There has been enough testing of homeopathy and plenty of evidence showing that it is not efficacious. Competition for research funding is fierce and we cannot see how further research on the efficacy of homeopathy is justified in the face of competing priorities. It is also unethical to enter patients into trials to answer questions that have been settled already.”

In the entire 275-page report, there is absolutely no mention of a research “ban”. WDDTY has taken the report’s clear, balanced and rational recommendation — to direct what limited funding is available towards new research that has at least a chance of yielding positive results — and turned it into “banning any further homeopathic research”: not just spin, but a complete lie.

(By the way, the report is a very good read for anyone sincerely interested in a clear, fair and logical analysis of homeopathy in the UK)

“It is depressing to see the interests of patients being threatened by a small posse of poorly informed and discourteous critics, who mix a little science with denigration and abuse”

The interests of patients are benefited, not threatened, by people highlighting that a popular treatment is ineffective. It’s no “small posse” of people who recognize the failure of homeopathy, and they’re not “poorly informed” – it’s practically the entire community of scientific and medical experts. And they don’t use “a little science” – they refer to the overwhelming consensus of the best available evidence.

WDDTY then claims that studies on animals and even plants have shown homeopathy treatments working better than placebos.

“All of this non-human evidence inherently rules out the dismissive ‘placebo-effect’ explanation.” / “homeopathic remedies have repeatedly been shown to have measurable effects on non-human biological systems, which inherently rules out the placebo effect”

No, it doesn’t. There are plenty of reasons to believe that placebo effects are in evidence in animals; not least that the effects of treatments are recorded by placebo-susceptible humans.

As for plants, this is arguably the low point of the article: WDDTY’s single stated reason to believe homeopathy benefits plants is a study in which 26 out of 30 homeopathic products failed; then when the test was repeated, three of the four remaining homeopathic products failed as well (in one of the preparations, the disease even got worse). And the single successful treatment (with “a small to medium effect size”)? A product for which the study’s authors state, “the exact composition … is not known” – in fact, it appears not to actually be homeopathic at all (a homeopathic version of that same product was one of the 29 that failed). WDDTY fails to mention any of these truths (as well as failing to mention that the journal the study is published in is notoriously shoddy). Citing this as favourable evidence is called p-hacking, and it’s notoriously deceitful (un)scientific behaviour.

To be as clear as possible: the data from this study suggest that homeopathy doesn’t work on plants, and WDDTY has used it (and as its sole reference) to claim that homeopathy does work on plants. It’s clear that WDDTY’s researchers are either completely incompetent or thoroughly dishonest, both to reference this journal and to misconstrue the information within it. Is this demonstrably dreadful magazine really a source that anyone would want to refer to or rely on?

“The war is not about evidence”.

The “war” – such as it is – is all about evidence. The evidence overwhelmingly shows that homeopathic remedies are nothing but placebos – if you’re prepared to look at all of the evidence, and separate the reliable evidence from the unreliable, rather than just referencing the (bad) studies that (do or can be spun to) support homeopathy.

“Because homeopathic medicines often contain not a single molecule of an active ingredient, opponents mock them as an affront to rationality—and indeed, a threat to the whole of science.”

Homeopathy isn’t “a threat to the whole of science” – the whole of science is a threat to homeopathy. If the evidence supported homeopathy, science would evolve to accommodate it. The evidence does not support homeopathy.(Here, for curiosity, is a non-exhaustive list of occasions when the scientific consensus moved on because of new evidence)

“Opponents make much of the ‘consistent failure to demonstrate effect beyond placebo’ when trials of homeopathy are studied; this is untrue.”

It only appears untrue if you accept poor evidence from bad studies, ignore good studies, and pretend anomalies are less statistically likely in good studies than they are. Which is to say, the claim that it’s untrue is itself fundamentally dishonest.

“However, after 70 years of being part of the NHS, homeopathy now faces extinction.”

Come on, no it doesn’t. Homeopathy is a booming business. (I admit this is speculation – I’ve found it extremely hard to come by reliable data on revenue or use of homeopathy; but I see no evidence that homeopathy is facing “extinction” or anything close to it. Boiron alone made a profit of €73.9 million in 2015)

“Clearly, both homeopathy and conventional medicine are on a par with each other when it comes to evidence-based medicine”

See the “bad data” section below for detail on why this is a lie.

“the NHS needs more homeopathy, say Britain’s 400 GPs who currently prescribe it.”

In this article, four – not 400 – GPs are quoted appearing to support homeopathy. As you’ll see in the “unsubstantiated claims” section below, GP referrals for homeopathy are “almost always at the patient’s request rather than as the result of a clinical decision”. It’s yet another outright lie to claim that “Britain’s 400 GPs who currently prescribe it [say] the NHS needs more homeopathy.”

This is WDDTY’s USP: helping people who believe in things disproven by science to convince themselves that they’re actually in good company, and that experts, evidence and science support their beliefs. In reality it’s lie after lie after lie.

“such stories [are] not uncommon”

It seems they are uncommon, since in a 3,700-word piece, WDDTY has only managed to find four GPs apparently prepared to tell such stories. (We know now that that’s because most of the 400 GPs WDDTY mentions probably don’t have such stories at all – they are, in fact, uncommon.) On top of this, I’m giving WDDTY the benefit of the doubt and assuming that the quotes from GPs are real; as you’ll see in the footnote at the end of this document, WDDTY is perfectly happy to make up quotes to support the most despicable lies.

“homeopathy—with its zero side-effects and arguably adequate reliability”

A serious examination shows that, despite trying really hard to do so, this article completely fails to demonstrate that homeopathic remedies have “arguably adequate reliability.” Indeed, the fact that the writers have to resort to the lies, misinterpretations, ad hominems and fallacies that they do suggests (correctly, as it happens) that there isn’t any good evidence that homeopathy has even arguably adequate reliability.

 

Referencing bad data/bad science

“An analysis of the responses to the homeopathic medicines revealed…”

The audit cited here (Tom Robinson in 2006) bears little resemblance to a scientific study. There’s no control, no placebo, no blinding. It’s not a study at all, just another series of anecdotes.

“Only 3 per cent of his [homeopathy] patients found their condition worsened following treatment”

Pointing out that ineffective remedies don’t usually make things worse is redundant. It’s not a point in favor of homeopathy.

“[Critics’] attacks echo those of the medieval Vatican against Galileo: it cannot be true, so it’s not.”

“It cannot be true, so it’s not” is a perfectly sound argument. It was wrong in the case of Galileo (if anyone even said it to him) because “it cannot be true” was a false claim. In the case of homeopathy: it cannot be true. So it’s not. (Also, and far more significantly: when it’s tested, it’s shown not to be true.)

By the way, the “Galileo Gambit” is a famous (and famously bad) debating technique.

“[Homeopathy’s] ultra-low doses are prepared by ‘serial dilution’ in water—adding a drop of the original medicine to 100 drops of water … homeopaths not only say such extreme dilutions work, but actually work better than the original medicine.”

Even WDDTY’s own writers appear not to have understood the basic theory of homeopathy. There is no “original medicine” to be diluted; only a substance that causes similar symptoms to those the patient is suffering.

“By the end of 2014, homeopathy had been tested in 104 RCTs for 61 different medical conditions: 41 per cent were positive; 5 per cent were negative; and 54 per cent were inconclusive. This track record is strikingly similar to conventional medicine’s. A 2007 analysis of a “large random sample” of RCTs of conventional treatments revealed that 44 per cent were ‘likely to be beneficial’, 7 per cent were ‘likely to be harmful’ and 49 per cent were inconclusive”

This statement about homeopathy studies is not, however, from a proper scientific review. It’s a claim from the Faculty of Homeopathy. WDDTY is comparing a peer-reviewed paper in the prestigious Journal of Evaluation in Clinical Practice to a dubious claim from the Faculty of Homeopathy. Putting the two in the same space and trying to draw a comparison is amateurish. Luckily the Lancet has performed a fair, direct comparison. Homeopathy came out worse.

Indeed, when serious meta-analyses are performed, it’s shown that “there are no health conditions for which there is reliable evidence that homeopathy is effective.”

Most of the studies WDDTY references appear to be any of:

  1. not scientific studies at all, just more anecdotes
  2. lacking controls, lacking blinding, lacking placebos
  3. performed on worthlessly small sample sizes
  4. published in poor quality journals – or even not published in a journal at all
  5. demonstrating different (sometimes opposite!) results to the results WDDTY claims.

As ever, they’re all cherry picked to support WDDTY’s position, ignoring the bigger picture: that the results of hundreds of studies – the vast majority of good studies – overwhelmingly show homeopathy doesn’t work.

And a word about animal studies: animal studies, like case reports and case series (and like anecdotes), are first steps. They’re dipping your toes in the water to see if it’s warm. You use their results to determine whether further, human studies are worth running (and safe to do so). To reference an animal study with positive results when human studies have already shown negative results is thoroughly dishonest (just as referencing individual studies when there are meta-analyses demonstrating the opposite results is dishonest). It’s cherry picking, again. WDDTY’s writers are either incompetent not to know this, or dishonest to ignore it.

More on the hierarchy of scientific evidence can be found here.

 

Straw man fallacies

“Only a very few critics confine themselves to what they regard as scientific principles—people who believe that science knows everything about everything, and nothing remains to be explained—scientific ‘fundamentalists’, perhaps.”

There is literally nobody in existence “who believes that science knows everything about everything, and nothing remains to be explained”. The threat of “scientific fundamentalism” continues to be massively overstated. Pretending that the people highlighting homeopathy’s flaws are simple-minded “fundamentalists” is a cheap straw man. See the rhetoric section below for more on WDDTY’s attempt to paint a childish caricature of its opponents.

 

Deceitful rhetoric, ad hominems, and hyperbole

WDDTY pretty shameless in their use of absurdly melodramatic language (some of these are quotes that WDDTY simply features, but quotes that they fail to challenge):

  • “propaganda campaign”
  • “The Good Thinking Society (a name eerily close to Orwell’s dystopian “Ministry of Truth”)”
  • “attacks echo those of the medieval Vatican”
  • “tide of hostility”
  • “vitriolic opponents”
  • “the supposedly respectable British Medical Journal”
  • “that arch-critic of homeopathy”
  • “a small posse of poorly informed and discourteous critics”
  • “scientific ‘fundamentalists’”

This shouldn’t be necessary in a serious scientific publication that can rely on the strength of its arguments. But when you strip out the misinformation, this childish rhetoric is all WDDTY has left. Of course, painting a paranoid picture of people who don’t share your beliefs as one-dimensional, conspiratorial villains makes it far easier to dismiss those people and the valid points they make.

 

Missing references (that is, unsubstantiated claims)

This is just bad practice: another reason to steer clear of the magazine (which on its Facebook page calls itself a journal; another lie):

  • “Homeopathic medicines appear particularly effective at the extremes of age…”
  • “Many GPs prescribe antibiotics for such conditions”
  • “[In Europe] studies have shown that primary-care practices offering homeopathy have better patient outcomes than pharma-only practices”
  • “One area where homeopathy has scientifically proved more effective than conventional drugs is with upper respiratory tract infections like flu, coughs, colds and sore throats.”
  • “an appropriate homeopathic remedy, now shown to be effective according to ‘outcome studies’”
  • “Britain’s 400 GPs who currently prescribe [homeopathy]”

I sought out the source of that final statistic, that 400 GPs (that’s all but 99.3% of them, by the way) currently prescribe homeopathy. The stat turned out to be yet another unsubstantiated claim from the Faculty of Homeopathy. Along my journey, I discovered that “a review carried out by West Kent NHS Primary Care Trust in 2007 found that [a referral for homeopathy] was almost always at the patient’s request rather than as the result of a clinical decision” (quote taken from Wikipedia; not the source, as the link was broken at time of writing). Even WDDTY’s attempts at appeal-to-authority fallacies are failures.

 

Non sequiturs coupled with hypocrisy and massive internal contradictions

“But over the past decade, conventional medicine itself has come under fire for having an equally poor evidence base…”

Conventional medicine allegedly having a poor evidence base is not an argument in favour of homeopathy (as the article itself admits moments later, thus demonstrating that this section was a waste of everyone’s time – except, again, for those people who will soak up the points presented and ignore the admission that they’re irrelevant). Ironically, WDDTY uses this section to suggest that scientific studies are practically worthless, then goes on to reference a load of scientific studies in homeopathy’s favor (two typical behaviors of proponents of flawed arguments: claiming evidence is unreliable except when it appears to be favorable; and trying to deflect attention to the flaws of others in order to avoid scrutiny of itself).

“Why should homeopathy be singled out?” asks Dr Kaplan. “To use evidence-based medicine to attack homeopathy exclusively and call for legislation against it, while huge swathes of conventional medicine lack evidence, is rather strange behaviour,” he says. “Double standards? That would be putting it euphemistically.””

Sure, if other treatments lack evidence, we should be acting against them too (and in many cases, we do, in contradiction of Kaplan’s lie that homeopathy is “exclusively” “attacked”). That’s not a reason to give homeopathy a free pass. Homeopathy gets so much attention focused on it because it’s so prominent. It gets attention focused on it because of all these anecdotes!

 

Misrepresenting information and quotes

“As a result, says Horton, “much of the scientific literature, perhaps half, may simply be untrue”.”

Logic of Science has addressed misuse of this quote specifically. Ironically the kind of papers that Horton suggests are “simply untrue” are the very ones that WDDTY repeatedly references. The unreliability of scientific papers is the very reason we strip out bad data using meta-analyses and systematic reviews – which are the very evidence that WDDTY is careful to ignore.

“That view is shared by the late Professor David Sackett, widely regarded as one of the fathers of clinical epidemiology: “Evidence-based medicine is not restricted to RCTs and meta-analyses,” he wrote 20 years ago. “It involves tracking down the best external evidence with which to answer our clinical questions.””

The best evidence. Not poor evidence cherry picked because it appears to support a position. And when the best evidence is RCTs and meta-analyses, we pay attention to them, rather than ignoring them in favor of poor quality studies and anecdotes. See my earlier link on hierarchy of evidence.

“Who said this? Ironically, no less a figure than that arch-critic of homeopathy, Edzard Ernst.”

(Interestingly in a related piece Ernst suggests that for some problems Acupuncture, Aromatherapy and Hypnosis can do more good than harm. Homeopathy isn’t on the list.)

These quotes are from people whom WDDTY vilifies when they disagree; but the magazine is happy to quote them when their words do (or can be spun to) support their position. Cherry picking. It’s cheap and dishonest and a mark of the gutter press.

 

The truth at last

“Opponents of homeopathy claim [outcome] studies are worthless as they cannot exclude either the placebo effect or the fact that many conditions are ‘self-limiting’—in time, patients get better anyway. The only way to establish whether homeopathy really works, they say, is through randomized controlled trials (RCTs) testing a homeopathic pill against an identical-looking placebo. According to British pressure group Sense About Science, “over 150 clinical trials have failed to show that homeopathy works. Some small-scale studies have yielded positive results, but this is due to poor methodologies or random effects.””

At last, WDDTY speaks the truth (broadly, with some exaggerated turns of phrase thrown in). These 90 words are just about the only content of any real value in the whole article.

 

Conclusion

In this article, WDDTY admits (correctly) that anecdotes don’t constitute good evidence, but spends most of its word count using anecdotes to back up its claims. It claims (falsely) that scientific studies are unreliable, then tries to back up its claims using scientific studies. The studies it does reference are awful. Sometimes those studies even draw the very opposite conclusions to that which WDDTY claims. WDDTY highlights an (alleged) poor evidence base for modern medicine in defense of homeopathy, then admits that an (alleged) poor evidence base for modern medicine does not constitute a defence of homeopathy. It entirely ignores the best available evidence, because that evidence doesn’t suit WDDTY’s argument — indeed, the best available evidence shows WDDTY’s position to be wrong. Finally, WDDTY makes a series of claims that are outright lies, a further series of claims that are unsubstantiated, and a series of suggestions that are childish, hyperbolic and absurd.

WDDTY doesn’t exist to tell the truth. It exists to tell its readers what they want to hear, and pretends to have evidence to back it up. With the deepest irony, I paraphrase their own accusation: “it’s not about evidence, but ideology.”

WDDTY’s staff are very good writers, who know how to cherry pick any apparent evidence that suits them (and omit that which doesn’t), dress it up to appear more reliable and balanced than it is – or even to make claims that it doesn’t really suggest at all – and throw in some incendiary hyperbole to excite its readers. I wish I could say the writers at WDDTY are just incompetent – but the high quality of their writing, and the fact that they admit their arguments are invalid and publish them anyway, suggests to me that the lies that fill their magazine are quite intentional. It doesn’t really matter — either way, it’s a thoroughly unreliable source.

There’s no global media conspiracy: if something in the media is important and true, you should almost always be able to find it cited by a credible source. If your best or only source is What Doctors Don’t Tell You (or Joseph Mercola, or Green Med Info, or Andrew Wakefield, or Natural News, all of whom use similar tactics), it’s not because those people and publications are beacons of truth trailblazing in a world of media corruption and deceit; it’s almost certainly because they’re wrong. And time after time, when their claims are scrutinized, they’re shown to be exactly that: wrong.

For a demonstrably rubbish article like this, WDDTY deserves to be avoided. When they publish vile, dangerous lies about the safety and effectiveness of lifesaving interventions like vaccines and cancer treatments (which they do, using the same cheap tricks that I’ve demonstrated above)*, anyone should be embarrassed and ashamed to have anything to do with them.

 

*Here’s one example. WDDTY, Jan 2013: “Chemotherapy isn’t only useless against cancer–it even encourages the tumour to grow, researchers have discovered… They say that chemotherapy is ‘completely worthless’ and that cancer sufferers would do better by avoiding the drugs altogether.”

The “completely worthless” quote is a fiction; the paper doesn’t recommend avoiding chemotherapy drugs; and when a skeptical reader subsequently contacted the paper’s authors for comment, they said: “The paper says nothing of the sort… Our study has been misquoted and misinterpreted – I believe on purpose…”

Posted in Vaccines/Alternative Medicine | Tagged , , , , | Comments Off on Debunking “The doctors case for homeopathy” by WDDTY: A case study in critically evaluating internet articles

Climate change and a flat earth (I can’t believe we’re still dealing with this argument)

Posted on December 16, 2016 by Fallacy Man
I'm not sure where this meme originated. if anyone knows, please tell me so that I can give credit.

Image via Terry Pratchett’s “Discovery World.”

When I started this blog, I wanted it to be entirely about science and the fundamental concepts of logic. As such, I decided that topics like religion and politics where off limits. However, anytime that a politician makes a statement about science, they enter my territory and I feel obliged to deal with their comments (by the same token, I only talk about religion when people use it to make claims about science). One of the most common and irritating ways that this occurs is when politicians try to compare modern science (usually climate research) to the idea that the earth is flat and/or we are the center of the universe. This argument is used ad nauseum and I have dealt with it before. Nevertheless, it is once again featuring prominently in our political dialogue, so I feel the need to explain why it is utter nonsense yet again.

I am, of course, referring to the recent comments by Trump advisor Anthony Scaramucci on CNN. When asked about the scientific consensus on climate change, he made several hopelessly ignorant comments like, “There was an overwhelming science that the Earth was flat, and there was an overwhelming science that we were the center of the world.” Statements like that are, however, absurd for numerous reasons.

First, and most obviously, they are blatantly false. Science is an extremely careful, empirical, and systematic process of making observations, forming hypotheses, and empirically testing those hypotheses by using repeatable, controlled* experiments, and that system is an extremely recent one. It has only existed in its modern form for the last few centuries. In contrast, we have known that the earth was round since the ancient Greeks! So there can’t possibly have been a scientific consensus that the earth was flat because science as we know it did not even exist then! Similarly, the idea that we are the center of the universe was discredited long before modern science, and the idea was largely based on religion. Indeed, although Galileo slightly predates modern science, when he demonstrated that the earth was not the center of the universe, he did so by applying the same basic type of systematic observations and reasoning that scientists use today. Thus, although modern science never said that the earth was the center of the universe, scientific reasoning was the thing that discredited that idea.

I want to elaborate on that last point for a second, because I want to be clear that I am not suggesting that there were no scientists before the modern era. There certainly were, but science was not the formalized discipline that it is now. In other words, although there were people who were making observations, testing hypotheses, etc., there was not a standard for the type of reasoning that qualified as science, nor was there the vast body of philosophical thought about the nature of science that we have today. My point is that the “science” that said that the earth is flat and the earth is the center of the universe is not actually the same thing that we know as science today, thus the comparison is invalid (i.e., it is an equivocation fallacy because it uses the word “science” to mean two different things).

Further, so far I have only been talking about the general concept of what science is, but we also have to take into account the tools that are available to scientists. For example, the statistics and mathematical formulas that we use today have only been around for about 100 years (sometimes much less than that). Those mathematics are extremely important because they are what allow us to rigorously examine our data and assign confidence values to our results. Similarly, today we can use computers to analyze millions of data points, run massive simulations, and do analyses that were unimaginable to scientists just a few decades ago. Further, our ability to accurately make the measurements that we use in those analyses has also increased astronomically. For example, we now have satellites that can continuously collect very precise data from all of the world. Tools like that give us incredible power for understanding our planet and the universe in general, and they give modern science a major advantage over past scientific endeavors.

There is also another problem here, and it is a more fundamental one. If this argument actually worked, then you could use it any time that you wanted, rather than simply for climate change. In other words, if the fact that people used to think that the earth was flat actually gave you carte blanche to reject science, then you would never have to accept any scientific result that you didn’t like. For example, I could say, “scientists say that washing your hands helps prevent the spread of disease, but they were wrong about the earth being flat, so I don’t have to listen to them when they say that we should wash our hands.” That is obviously absurd, but that is exactly the same thing that Scaramucci did. His argument was logically identical, which means that if either of those arguments worked, then both should work, and the fact that the hand washing example is clearly absurd means that the climate change argument must also be absurd. That’s how logic works.

Nevertheless, you might be tempted to think, “but scientists aren’t completely, 100% sure that we are causing climate change, so it is always possible that they are wrong.” Technically, that is true, but here is the important qualifier: science never gives us 100% certainty. It is inherently a skeptical process, and it only tells us what is most likely true given the current evidence, not what is absolutely true (details here). So sure, it is true that I can’t be 100% sure that we are causing climate change, but I also can’t 100% sure that the earth is round. Indeed, I can’t be 100% sure that you even exist, nor can I be 100% sure that Donald Trump isn’t an alien squid from another universe. However, the fact that I can’t be 100% sure about those things does not mean that I get to have any form of practical doubt about them. It would clearly be ridiculous for me to actually think that Trump is an alien squid, and politicians should not make laws based on that notion. Even so, the fact that we can’t be 100% sure about climate change does not mean that you get to have any practical doubt about it, nor should we be basing laws on that doubt.

In technical terms, what I have been describing is known as an argument from ignorance fallacy. It can occur in many ways, but one of its forms happens when you say, “we can’t be sure that X is true, therefore X is false” or “we can’t be sure that X is true, therefore it is reasonable to believe that it isn’t true.” This is not logically valid. The fact that we can’t be completely certain of a conclusion does not mean that you can assume that all of the evidence supporting that conclusion is false. This comes back to one of my favorite topics: the burden of proof. The person making a claim is always responsible for providing the evidence to support that claim. In other words, if you want to claim that we aren’t causing climate change, then you must provide actual evidence that we aren’t. You can’t simply assert that we can’t be 100% certain, the models might be wrong, etc. If you want to claim that all of the scientific evidence is wrong, then you have to provide evidence to support that claim, not speculations. Further, although it is true that science never proves anything with 100% certainty, there are some topics that are so well supported by so many lines of evidence that it is almost unthinkable that they could actually be totally wrong, and anthropogenic climate change is one of those topics (others include atomic theory, the germ theory of diseases, etc.).

This brings me to the final topic that I want to discuss here: the actual evidence for climate change. You see, so far I have been talking in pretty general terms about science, but Scaramucci’s statements are particularly absurd and misguided when we focus in on climate change, because there is so much evidence supporting it. As I have explained in more detail elsewhere, we have tested the sun, volcanoes, and other natural drivers of climate change, and they can’t explain the current warming by themselves (Meehl, et al. 2004; Hansen et al. 2005; Wild et al. 2007; Lockwood and Frohlich 2007, 2008; Lean and Rind 2008; Foster and Rahmstorf 2011; Imbers et al. 2014). However, including our greenhouse gases in the calculations does explain the warming trend (Stott et al. 2001; Meehl et al. 2004; Allen et al. 2006; Lean and Rind 2008; Imbers et al. 2014). In other words, we know that our emissions are driving the warming, because we have tested the natural drivers of climate change, and none of them (even when combined) can account for the current increase in temperatures. Further, we know that CO2 traps heat and is largely responsible for both our climate and past climate changes (Lorius et al. 1990; Tripati et al. 2009; Shakun et al. 2012), we know that we have greatly increased the CO2 in the atmosphere (Bohm et al. 2002; Ghosh and Brand 2003;Wei et al. 2009), and we have used satellites to directly measure the amount of heat energy that CO2 in our atmosphere is trapping, and (just as expected) it is trapping an increasing amount of heat (Harries et al. 2001; Griggs and Harries 2007). In other words, we have direct evidence that our CO2 is causing the earth to trap more heat (i.e., warm). The evidence is overwhelming, and when you look at the scientific literature, you will find thousands of papers saying that we are causing climate change, and only a tiny handful that disagree. The scientific consensus on this topic is incredible (details here), and the consensus among scientists exists because of the evidence. Scientists do not “believe” in climate change (as Scaramucci erroneously asserts), rather we accept it as fact because of the overwhelming evidence. There is simply no serious debate on this topic among actual scientists.

To illustrate this a different way, a few weeks ago, I attended the annual meeting of the Ecological Society of Australia, and there were lots of talks about the effects that climate change is having and will have on plants and animals, but not one of those talks hedged their comments with statements like, “if climate change is real.” Rather, all of them treated anthropogenic climate change as a scientific fact because that is what it is, and it is absolutely crazy to suggest that we should act as if it isn’t settled just because some non-scientists over 2000 years ago thought that the earno matter what crackpot notion you believeth was flat! Further, the fact that a handful of scientists today disagree doesn’t help that argument either because there will always be a few who disagree on any topic (this is an appeal to authority fallacy). Having an advanced degree does not guarantee that you know what you are talking about, and you can sadly find plenty of crackpots even among scientists (note: before you accuse me of committing an appeal to authority fallacy by citing the consensus, please read this post on the difference between appealing to authority and deferring to experts).

To be clear, there is disagreement about the extent of climate change that we are going to cause. There are lots of variables that have to be considered, and there are admittedly still topics that we are actively studying in order to refine out models. However, although the models may be imprecise about the extent of the warming, it is almost unthinkable that we are wrong about the warming itself. Indeed, the idea that we are causing the climate to change is not based on the predictive models, rather it is based on research like the papers I presented earlier. Further, out models have actually done a pretty good job of predicting the current warming, so although they aren’t perfect, they probably aren’t that far off (Hansen et al. 2006; Frame and Stone 2012; Rahmstorf et al. 2012; Cowtan et al. 2015; Marotzke and Firster 2015).

In short, the science that said that the earth is flat or that we are the center of the universe was not the same thing as what we know as science today. Further, the past few decades have furnished us with a fantastic array of amazing tools for collecting and analyzing massive amounts of data, and when we apply those tools to the topic of climate change, we get an extremely consistent result: it is real and we are the main cause of it. Because of that evidence, there is no serious debate about this among scientists. Yes, there are a few dissenting voices, but those will always be there. Further, yes, it is always technically possible that we are wrong, but it is also technically possible that Donald Trump is a space squid, and the fact that something is technically possible does not mean that it is rational to think that it is actually true.

*Note: When I said that science uses controlled experiments, I was not referring specifically to randomized controlled trials (RCTs). Although they are a particularly powerful design, they are far from the only one, and they often aren’t applicable in fields outside of medicine. In climate research, for example, we generally can’t do RCTs, but we can still control our experiments and analyses by measuring all of the confounding factors and including those factors in out analyses.

Related posts

Literature cited

  • Allen et al. 2006. Quantifying anthropogenic influence on recent near-surface temperature change. Surveys in Geophysics 27:491–544.
  • Bohm et al. 2002. Evidence for preindustrial variations in the marine surface water carbonate system from coralline sponges. Geochemistry, Geophysics, Geosystems 3:1–13.
  • Cowtan et al. 2015. Robust comparison of climate models with observations using blended land air and ocean sea surface temperatures. Geophysical Research Letters 42:6526–6534.
  • Foster and Rahmstorf 2011. Global temperature evolution 1979–2010. Environmental Research Letters 7:011002.
  • Frame and Stone 2012. Assessment of the first consensus prediction on climate change. Nature Climate Change 3:357–359.
  • Ghosh and Brand. 2003. Stable isotope ratio mass spectrometry in global climate change research. International Journal of Mass Spectrometry 228:1–33.
  • Griggs and Harries. 2007. Comparison of spectrally resolved outgoing longwave radiation over the tropical Pacific between 1970 and 2003 Using IRIS, IMG, and AIRS. Journal of Climate 20:3982-4001.
  • Hansen et al. 2005. Earth’s energy imbalance: confirmation and implications. Science 308:1431.
  • Hansen et al. 2006. Global temperature change. Proceedings of the National Academy of Sciences 10314288–14293.
  • Harries et al. 2001. Increases in greenhouse forcing inferred from the outgoing longwave radiation spectra of the Earth in 1970 and 1997. Nature 410:355–357.
  • Imbers et al. 2014. Sensitivity of climate change detection and attribution to the characterization of internal climate variability. Journal of Climate 27:3477–3491.
  • Lean and Rind. 2008. How natural and anthropogenic influences alter global and regional surface temperatures: 1889 to 2006. Geophysical Research Letters 35:L18701.
  • Lockwood and Frohlich. 2007. Recently oppositely directed trends in solar climate forcings and the global mean surface air temperature. Proceedings of the National Academy of Sciences 463:2447–2460.
  • Lockwood and Frohlich. 2008. Recently oppositely directed trends in solar climate forcings and the global mean surface air temperature. II. Different reconstructions of the total solar irradiance variation and dependence on response time scale. Proceedings of the National Academy of Sciences 464:1367–1385.
    Lorius et al. 1990. The ice-core record: climate sensitivity and future greenhouse warming. Nature 139–145.
  • Marotzke and Firster 2015. Forcing, feedback and internal variability in global temperature trends. Nature 517:565–570.
  • Meehl, et al. 2004. Combinations of natural and anthropogenic forcings in the twentieth-century climate. Journal of Climate 17:3721–3727.
  • Rahmstorf et al. 2012. Comparing climate projections to observations up to 2011. Environmental Research Letters 7:044035.
  • Shakun et al. 2012. Global warming preceded by increasing carbon dioxide
  • Stott et al. 2001. Attribution of twentieth century temperature change to natural and anthropogenic causes. Climate Dynamics17:1–21.
  • Tripati et al. 2009. Coupling CO2 and ice sheet stability over major climate transitions of the last 20 million years. Science 326:1394–1397.
  • Wei et al. 2009. Evidence for ocean acidification in the Great Barrier Reef of Australia. Geochimica et Cosmochimica Acta 73:2332–2346.
  • Wild et al. 2007. Impact of global dimming and brightening on global warming. Geophysical Research Letters

 

Posted in Global Warming | Tagged , , , , | 18 Comments