For the first question, can experiments detect differences that matter, this has significance in interpreting scientific literature because if there are no significant differences detected in experiments, that those experiments don’t really matter as much. This question is critical in interpreting scientific literature because it’s a good weed-out question for deciding which published papers really matter and which ones are just feeding more into the hype of a new field. Hanage also discusses the importance of remembering that “apparent similarities might cloak important differences”, which I think is important to keep in mind so we don’t jump to conclusions based off of one piece of scientific literature alone.
For the second question, does the study show causation or correlation, this has significance in interpreting scientific literature especially because a large majority of recent scientific literature on the microbiome focuses on whether certain aspects of the microbiome are linked to disease or not. By differentiating between whether a study shows causation or correlation, we can differentiate between confusion and hype vs. genuine findings in scientific literature.
For the third question, what is the mechanism, this has significance in interpreting scientific literature because of its relation to the previous question: as Hanage discussed, “correlation isn’t causation, but correlation almost always implies some sort of causal relationship. We just don’t know what it is.” Asking what the mechanism is has significance in interpreting scientific literature because it’s important to be understanding how authors of articles are coming to their conclusions since this influences the validity of their conclusions.
For the fourth question, how much do experiments reflect reality, this has significance on interpreting scientific literature because we have to remember that not all experiments are going to reflect reality. The lab can be a lot different from the real world; there are things that can cause experimental effects but aren’t a key cause of what’s seen outside the lab. This question is also extremely significant in interpreting scientific literature because due to ethical issues, researchers can’t experiment on other humans. This leaves them to test experiments on other animals (usually mice) and apply these results to humans, but it’s important to remember that not all experiments can be generalized in that way.
For the fifth question, could anything else explain the results, this has significance on interpreting scientific literature because there are often confounding factors that can greatly influence the results. We as the readers won’t see those factors, and the researchers themselves might not always see those factors either. It’s important to remember, especially in literature from scientific fields with a lot of hype and buzz from the general public, that there could always be something else causing the results seen. Hanage describes this as “resisting the urge to transform our microbial passengers into modern-day phantoms.”
I think that the question most helpful in discussing controversy is “could anything else explain the results” because it asks people to consider if there are contributing outside factors that the experiment didn’t look at that could be causing results that might not make much sense. It could eliminate controversy around certain scientific subjects if there were other factors that had a large influence over the results.
