- Can experiments detect differences that matter?
This is important in interpreting scientific literature because data must be scientifically significant to indicate a relationship between two things. If the difference from the control is tiny, that does not reveal anything to us.
2. Does the study show causation or correlation?
This is especially important in interpreting scientific literature because it is very easy to find a correlation between things if you are looking for it. There may be a correlation between people that like ice cream and people that like coffee, but liking ice cream doesn’t cause someone to like coffee. Causation shows that there is a direct link between two factors.
3. What is the mechanism?
It is important to know the mechanism of how related things work because it often reveals to us that there are many more factors playing a role than just the ones you are looking at and it often opens the door for more research. It isn’t enough to know there is a correlation between two things, we want to know how one influences the other.
4. How much do experiments reflect reality?
Proper experimentation is what gives scientific work validity. If an experiment proves something but can’t be reflected in real life it means nothing to us. We need an experiment to be as close to reality as possible so that when it is applied to humans from mice for example, we know the relationship they are trying to prove is accurate.
5. Could anything else explain the results?
Confounding factors exist in nearly everything we study. It is very rare to find a relationship between two factors that is not influenced by something else. What you are studying may be the primary reason, but it is likely influenced by something else as well. For example, weight loss is not only influenced by the amount of calories you eat. Genetics, sleep, and many other things influence it as well, so it is vital in scientific research to look at what else might be causing results. This is why controls are so important so you make sure you are looking at the factor of interest specifically.
I would argue that the most important in discussing controversy is “What is the mechanism?” This may not be the most obvious first choice but I think knowing the mechanism of how something works displays the most knowledge about that topic. If you know the mechanism, you can see where confounding factors might exist, you can determine correlation vs causation, and you can argue ways that changing this factor would influence the outcome. I think knowing the mechanism answers almost all of the other questions. It is the hardest thing to determine and therefore requires the most evidence to support it which is essential when discussing controversies.
