So in response to Catseye’s question about going straight to the Academic Horse’s Mouth when researching one’s ailments and one’s doctors’ schemes — “can the average Joe understand what they’re reading? It sounds intimidating, to say the least” — I said I’d write a post on how to read a piece of medical (or any other scientific) research.
The answer to that question is YES! Most people can understand enough of a research paper to pick up on the important points. And it only sounds intimidating. It is surprisingly, weirdly easy to understand most published technical research papers.
Here’s the secret: YOU DON’T HAVE TO READ THE WHOLE DAMN BRAIN-BANGING THING!
To understand what the researchers are trying to do and what they’ve found out (if anything), you really only need to read about a third of their paper — and that is the most accessible third. What you should know is that scientific papers follow a standard format. They’re always divided into these sections:
- Abstract: A brief summary of the project & findings — very brief.
- Introduction: Description of the background, purposes, and design of the project. Usually contains a Statement of the Problem: an explicit, carefully worded explanation of the issue, in short. Sometimes this section will also contain what is called a “review of the literature,” in which the authors reprise the high points of previous work.
- Methods: Explanation of their approach to the study and the tools or strategies they used in going about the research.
- Results: Description of what happened when they applied the Methods to the research problem. This section may contain graphs and tables that summarize the study’s findings.
- Discussion: Addresses the results and their implications in light of what is already known. This section may also contain any caveats about what remains to be found, drawbacks to the research, and what further research needs to be done.
- Conclusion: Sometimes suggestions for further research appear in a separate section, usually called “Conclusion.” The conclusion is often presented together with the Discussion section.
- References/Bibliography: A list of the published sources used in the paper. This is useful to you because if it contains a lot of flakey sources, you’ll know the paper itself is probably flakey. If it contains substantial sources from established researchers and credible institutions, you can base your assessment of the authors’ credibility partially on their sources.
Before you even begin to read the paper, first determine the value and credibility of the journal or book publisher that has issued the thing. Ideally, you would like a paper to have appeared in what is called a tier-1 journal — i.e., at the top of the profession. But that is not always possible — some excellent work appears in lesser publications. Look at the title of the journal. If it is well known or obviously the emanation of a highly ranked university or research organization, then you can feel some confidence in it. Examples: New England Journal of Medicine. Journal of the American Medical Association. The Lancet. The British Medical Journal. The Centers for Disease Control. Johns Hopkins University. Stanford University Medical Center.
And so on.
Here is a list of journal rankings in medicine. Bookmark this page and check your sources against it.
Be aware that the woods are full of fake academic journals. These are called predatory journals: phony or extremely low-ranking journals that charge academic researchers for the privilege of publishing third-rate (or less) work in their shoddy pages. They exist because young scholars must publish to obtain promotions in academia; often they must already have published even to get a job. Usually these frauds have convincing, official-sounding titles. Here is a more or less up to date list, based on the late, great Beall’s List.
A legitimate journal is peer-reviewed. This means everything it publishes is read, critiqued, and assessed by experts in the subjects the journal addresses. To be published in such a journal, an article must pass peer-review. In other words, it must have at least some semblance of quality, credibility, and accuracy.
Beall’s list used to keep tabs on predatory journals. One day it was yanked off the air. Gossip has it that the proprietor was threatened with a lawsuit by a combine of the crooked journals he listed. So, this valuable resource no longer exists in its full glory. For a hint at the ridiculous scamminess of fake journals, take a look at this highly entertaining article on their practices in hiring “editors.”
Sometimes if you look up a journal title in Wikipedia, the article will mention, in a mealy-mouthed way, if the publisher has ever been accused of predatory practices. But that is not 100%. Try to stick to the old standards,.
Okay. So once you’ve found an article in a journal you think is credible, here’s what you’re going to read, in this order:
- Abstract
- Results
- Discussion/Conclusion
- Tables/graphs (if any)
That’s pretty much it. If you feel inclined to plow through other sections, you can. But the information you really need appears in the sections above. Often the results are summarized well enough that there’s no reason to pore over the data in the tables and graphs.
Where can you go to find these publications?
A Google search will bring up some of them, if you enter the right key terms.
Google Scholar will bring up a greater percentage of true scientific papers. Google Scholar, however, tends to be out of date.
A college or university library has databases that contain subscriptions to journals, and so the contents are wider, deeper, and timely. Some major metropolitan libraries also provide access to these resources. You don’t have to be a student or employee of an academic institution to get access to its library’s databases. Most college and public university libraries will provide a library card — for a fee — to members of the public.
What about all those plain-English websites, the ones that often come up at the top of a Google search?
Well, for basic needs, they can suffice. The best of them are published by hospitals and medical centers. But…caveat emptor…
- Sometimes they’re very much dumbed down.
- Sometimes they support an agenda.
- Sometimes they’re published by associations and nonprofits supported by Big Pharma or other self-interested parties.
- Usually they present the received wisdom — they echo what your doctor will tell you, which may or may not be at the cutting edge.
- Sometimes they’re…well…bullshit.
Always take “alternative medicine” websites with a very large grain of salt. If you’re gonna go in for alternative medicine, there’s really no point in wasting your time trying to understand hard science — you’re taking a leap of faith, and you might as well accept that for what it is. Faith, not science.
That’s OK, if it suits your temperament. My mother’s family were Christian Scientists. Two of them lived into their mid-90s and never saw a doctor in their lives. If that works for you, then it works for you. But…don’t imagine “alternative medicine” is based on scientific research. It is not.
Watch out for any site peddling the advice and opinions of “Dr. [Firstname].” Anyone who addresses you in this way, pretends to be a celebrity, or presents information in talk-show, folksy, People-Magazine style format is a showperson, not a scientist. Advice appearing at these sites is usually cursory, dumbed-down, and incomplete.
There ya go: that’s about all you need to know.
