• Can experiments detect differences that matter?
• Does the study show causation or correlation?
• What is the mechanism?
• How much do experiments reflect reality?
• Could anything else explain the results?

The questions above are all important questions when reading, writing, or performing scientific research.

One, it is important for research to detect differences that matter. This is very important as it is really the point of research, to find differences between groups and discover what causes the difference.

Two, it is important to know if a study is pointing to causation or correlation. In a lot of observational studies we can discover correlation, but we can never say there is causation (yes, I learned this in stats… Thank you Jeff).

Three, what was the mechanism that was used to create this different? After a pattern is recognized the next step is to figure out what is creating it and how.

Four, when reading scientific articles it is crucial to decipher how representative the study is of the actual environment. For example does an in vitro study represent what would happen in a human body?

Five, is the result we are seeing caused by what we think it is? In some studies there are too many variables at play to confidently say that A causes B. This is why a good experimental design is important, so that variables can be removed from the equation.

All of these are important, but as for which one is the most helpful for discussing controversies I don’t know if there is an answer that fits every situation. I truly think it depends on the study. I think generally the last two are the most applicable though. Because often studies are too broad and their results could have been caused by an number of things, or they are to narrow don’t represent reality. Really though all of these are important when reviewing science and for interpretation.