Find primary research literature baffling? Violent Metaphors has step-by-step instructions on how to cut through the jargon and interpret experimental scientific findings for yourself:
Before you begin reading, take note of the authors and their institutional affiliations. Some institutions (e.g. University of Texas) are well-respected; others (e.g. the “Discovery Institute”) are actually agenda-driven.
As you read, write down every single word that you don’t understand. You’re going to have to look them all up.
Begin by reading the introduction, not the abstract.
Identify the BIG QUESTION. Not “What is this paper about”, but “What problem is this entire field trying to solve?”
Identify the SPECIFIC QUESTION(S) What exactly are the authors trying to answer with their research? What are the authors going to do to answer the SPECIFIC QUESTION(S)?
Now read the methods section. Draw a diagram for each experiment, showing exactly what the authors did. Include as much detail as you need to fully understand the work.
Read the results section. Don’t yet try to decide what the results mean, just write down what they are. You’ll find that, particularly in good papers, the majority of the results are summarized in the figures and tables.
THINGS TO PAY ATTENTION TO IN THE RESULTS SECTION: Any time the words “significant” or “non-significant” are used. These have precise statistical meanings. If there are graphs, do they have error bars on them? For certain types of studies, a lack of confidence intervals is a major red flag. The sample size. Has the study been conducted on 10, or 10,000 people?
What do the authors think the results mean? Do you agree with them? Can you come up with any alternative way of interpreting them? Do the authors identify any weaknesses in their own study? Do you see any that the authors missed?
What do other researchers say about this paper? Who are the (acknowledged or self-proclaimed) experts in this particular field? Do they have criticisms of the study that you haven’t thought of, or do they generally support it?
Read the rest at Violent Metaphors