Prompt:
In W. P. Hanage’s article, he discusses the importance of five key questions when interpreting scientific literature:
- Can experiments detect differences that matter?
- Does the study show causation or correlation?
- What is the mechanism?
- How much do experiments reflect reality?
- Could anything else explain the results?
Explain the significance that each of these questions have on interpreting scientific literature. Which is most helpful when discussing controversy, and why?
Response:
- The significance of this question is that, as humans, we have yet to reach the ability to fully understand genomes within microbiomes to detect differences that truly matter. The microbiome is hugely diverse and expansive, and it is difficult to observe all the changes from an experiment across the entire microbiome. Hence, we look for things that are “supposed” to change. Other factors could also change as well, and these could be even more important than what the experiment is testing. Hence, it is important to note that not all experiments are able to fully understand the extent of microbiome and not all the necessary information may be present.
- The significance of this question is that people need to explore the study done before taking the study’s word. The example giving within the paper perfectly describes that while the paper stated it was a “causation”, it was most likely a “correlation”. Other factors need to be considered when reviewing an experiment, especially if it can directly impact the experimental question.
- The significance of this question is that now that technology has advanced in such a way that allows us to pick out a taxa and see what that specific bacteria does. It is important for studies to be more precise, and not speculate on a broad range of bacteria. Having a study be more precise about a taxa or phyla is more convincing that it will have a said effect on the experimental question. Everything can have a casual relationship, and thus experiments should be exact.
- The significance of this question is that a lot of primary research articles are done on mice, and more often than not, they are germ free mice. This is certainly not modeled in humans. Hence, the study can be completely irrelevant to humans and this is important to consider. Humans are more complex and all come from different backgrounds that cannot be controlled.
- The significance of this question is that it refers to a lot of built up hype and also things that are beyond researchers controls. Therefore, when reading an article, the study needs to be big enough that it is actually a significant result and does not happen by chance. There are also needs to be a lot of controlled variables in the experiments. Additionally, those who write about the studies in a news article may not correctly interpret the results and hype up the article inappropriately.
I think the question that is most helpful for discussing controversy is, “how much do experiments reflect reality?”. I think this one does a good job of actually tackling the elephant in the room, is this even an applicable study? A lot of experiments are done in such an environment that can never be modeled for an actual human or environmental scenario and therefore is completely irrelevant.
