I spend a lot of time on this blog talking about scientific studies and how to analyze them, but there is a very important topic that, until now, I have only mentioned briefly: abstracts. Abstracts are intended to be useful tools to help scientists decide whether or not to read a given paper, but they are often misused and abused. Indeed, it is extremely common for people to base an argument entirely on an abstract. Even more disturbingly, I often see people citing conference abstracts as evidence. On numerous occasions, I have seen someone claim that, “A new study found X,” and when I follow the link, the study that they are referring to was presented at a professional conference, but has yet to be peer-reviewed.

There is a huge difference between reading a study and reading the study’s abstract

To be clear, this is in no way a problem that is specific to those who routinely deny well established science. Rather, this is a problem that is rampant among staunch supporters of science as well. Therefore, in this post, I want to explain what abstracts are actually intended for, how you should use them, and how you shouldn’t use them. I also want to spend some time specifically on conference abstracts, because they are a special case that deserves extra attention.

The real purpose of abstracts
To begin with, I want to briefly explain what abstracts are actually intended for. Their purpose is simply to help researchers decide whether or not it is worth their time to locate and read the entire paper. This is a very useful function because critically reading scientific papers takes a long time. I’m a fairly fast reader, but if I want to really understand a paper, and go through all of the proper checks to make sure that it was done correctly, it will take me a minimum of one hour to read it (usually a lot more). So, given the millions of papers that have been published, a single scientist clearly can’t read all of them. Even within their chosen discipline, there will generally be far more papers than they could realistically read.

This is where the abstract comes in. A well written abstract will contain a very brief explanation of the background and importance of the study, a brief explanation of how the study was conducted, and the key results of the study. Scientists will then use that information to determine whether or not that paper is relevant for the question that they are working on. That’s it. That is the purpose of an abstract. It is to help scientists maximize their efficiency by focusing on papers that are relevant to them. It is not intended to be a through summary that you can read instead of reading the actual paper.

Think of abstracts like the descriptions on the back of a DVD case. You aren’t supposed to read the description instead of watching the movie. Rather, you are supposed to use the description to decide whether or not to watch the movie, and it would be silly to enter a debate about a movie for which you have only read the description. Similarly, you should use abstracts to determine which papers are worth reading, but you shouldn’t read the abstracts instead of reading the papers themselves.

Why you shouldn’t trust abstracts
The common tendency to read abstracts instead of the actual studies is extremely problematic because you can’t do any of the normal quality checks with just an abstract. Unfortunately, not all scientific studies are conducted properly. There is a lot of junk science out there. So, whenever you read a scientific study you need to very carefully look at the experimental design to make sure that it was set up correctly (e.g., was it properly randomized, did it use proper controls, etc.), you need to look at the statistics to make sure that they were done correctly, you need to look at the conclusions to make sure that they are justified by the results, etc. None of that is information that you can get from an abstract. So the abstract may look great, but the study itself might be fundamentally flawed. For example, last week I dissected a recent study that claimed that Splenda causes cancer in mice, and most of the things that I commented on were things that I could not have assessed from just the abstract. If all that I read was the abstract, I never would have seen things like the questionable statistics or inconsistent results.

This brings me to my next major point: abstracts often misrepresent studies. It is very easy to write a misleading abstract. Because abstracts are brief, they generally only highlight key results, but this means that authors have the opportunity (whether intentionally or unintentionally) to only present the results that support their conclusion while ignore results that challenge it. Similarly, abstracts generally include a general conclusion statement from the authors, but I have seen many, many cases where that conclusion statement really wasn’t supported by the data in the paper. To put this another way, abstracts often reflect the authors’ opinions about their study rather than the actual results of the study.

Finally, abstracts, like DVD cases, are designed to make you want to read the rest of the study (or watch the movie). They are scientists’ opportunity to advertise their research, and this sometimes results in abstracts that oversell the significance of the research or gloss over serious problems. To make another movie analogy, it would be silly to judge a movie by its trailer. Even so, you shouldn’t judge a paper by its abstract.

All of this simply means that you need to read the study itself before you either cite it as evidence or conclude that it is not good evidence. Reading the abstract is not enough to tell you what the study actually found or if you should even accept its results.

Understanding conference abstracts
Scientific conferences are lots of fun. I always look forward to them because I get to meet my fellow scientists (often including the great minds of my field), as well as presenting my own research and learning about other people’s research (btw, you can go to these meetings even if you aren’t a scientist, and if you are really interested in science, I highly recommend it). In a situation that is similar to the scientific literature, however, most meetings have more talks than one person can attend (there are usually several sessions running concurrently). So, much like the abstracts for papers, conference abstracts are simply intended to help people figure out which talks they want to see. Thus, they are very similar to the abstracts for papers, but there are a few key differences that make them even more dubious as sources of evidence.

First, conference abstracts generally aren’t peer-reviewed (there are a few exceptions, and some conferences even require a short paper, but that is not the norm). In other words, conference abstracts are usually accepted or rejected entirely based on whether or not they fit within the scope of the conference. This is extremely important because it means that other scientists have not meticulously examined the research to make sure that it is of a high quality. In other words, having an abstract accepted to a conference is in no way an affirmation that the research is solid or that the results are trustworthy. Even for the few conferences that review the abstracts, the review process is generally much less stringent than it is for publishing a paper in a journal.

Indeed, it is extremely common for the research in presentations/posters to never see the glorious light of publication, and if I am going to be totally honest, this was the case for one of my very first research projects as a lowly undergraduate. This is, in fact, commonplace for conference presentations. I have sat through some truly horrible talks, but the abstracts for those presentations generally looked great.

This brings me to me next major point. Remember earlier how I said that you need to critically analyze papers? Well you can’t do that with conference abstracts. This is the primary reason why they should not be used as evidence. When someone cites a conference abstract to back up their claim, all of that you have is that abstract. You have no way of checking the researchers’ methods, statistics, detailed results, etc. Which means that you have absolutely no way of knowing whether or not study was done correctly.

Further, talks and posters at conferences are often on projects that are currently in progress. As a result, they often include incomplete sample sizes and preliminary results. Thus, even when a talk does go on to be published in a reputable journal, the final data and conclusions in the paper often differ from the conclusions in the talk.

Similarly, another important function of conferences is actually to get feedback on your research prior to submitting it for peer-review. It is extremely common for the question and answer section of talks to include comments from other researchers such as, “Have you tried X?” or “You need to make sure that you account for Y.” These comments can be extremely helpful in refining your analyses and improving your research, but because they are always made during the conference, they do not get included in the abstract. In other words, the abstract is on the unrefined version of the data that has yet to receive casual feedback from your peers, not alone formal peer-review.

Advice on getting copies of papers
At this point, you may be thinking, “This is all well and good, but half the time the paper is behind a paywall, so all that I have is the abstract.” That is admittedly a real problem, but there are ways around it.

First, for conference abstracts you need to try to figure out whether or not the study has actually been published in a proper peer-reviewed journal. If it has, then you should read that study, and ignore the conference abstract. If it has not been published, the you should not be citing the abstract as evidence (sometimes a lack of publication is actually a sign that the paper was flawed).

As far as the actual papers, getting them can be challenging. The first thing to do is check both Google and Google Scholar. You can often get free copies of papers from them (note: most journals allow authors to self-archive their papers, so those pdfs often are legal). If you can’t get them there, you might be able to get them via your local library, especially if you have access to an academic library on a college campus. Even if you aren’t a student, many universities have a library that members of the public can visit, and those libraries usually contain a large periodical room where you can read and photocopy/scan articles from lots of different scientific journals.

Another option is simply to email the authors. You can usually get the lead author’s email address without actually paying for the paper (or if nothing else, Google them), and scientists will usually be more than happy to send you a free copy of their papers. So don’t be afraid to send them a brief, polite email asking for a copy of the paper. Preferably, include something like, “paper copy request” in the subject, and include the full title of the paper you want in your email. Don’t just say something like, “I want a copy of your paper on vaccines” because odds are that they have many papers on vaccines.

Conclusion
The take home message from this post should be self-evident, but I will state it anyway. Abstracts should not be used as evidence. They do not give you enough information to properly assess a study and they often reflect the authors’ biases rather than the actual results of the paper. Abstracts from conferences are even worse because they are often on preliminary data and generally have not undergone peer-review. Therefore, you need to read the actual study itself, not its abstract.

 Other posts on understanding scientific papers

• 10 steps for evaluating scientific papers
• 12 bad reasons for rejecting scientific studies
• Debunking “10 scientific studies proving GMOs can be harmful to human health”
• Does Splenda cause cancer? A lesson in how to critically read scientific papers
• Is the peer-review system broken? A look at the PLoS ONE paper on a hand designed by “the Creator”
• Peer-reviewed literature: What does it take to publish a scientific paper?