5 Questions for interpreting scientific literature:
- Can experiments detect differences that matter?
Hange was getting at the fact that it is hard to distinguish bacterial communities. If we use 16S genes to identify the species in a bacterial community, this does not tell us all the information about the microbes present. Hange argues that closely related genes can still have differences that we may not be able to interpret through sequence alone. If we cannot tell one from the other, we probably don’t have enough information to make a finite conclusion.
- Does the study show causation or correlation?
Correlation or causation seems like the ‘chicken and the egg’ of microbial communities. This is significant because as Hange says, we can jump to conclusions to explain our data without taking into consideration other variables.
- What is the mechanism
Hange argues that we need to use a ‘reductionist’ approach to studying microbiomes, because we need to know if/how the microbiome affects human health. Sometimes in microbiology the mechanism is unknown. However, it is important to ask what the mechanism is so that we know more about its interactions.
- How much do experiments reflect reality?
Hange shows examples of how mouse models are not able to exactly transfer to human models. Experiments may not apply to life outside the lab because experiments control for outside variables that may be very present in real life. It is important to ask how applicable the results of an experiment are.
- Could anything else explain the results?
Hange concludes that it is normal for people to think there is something they can do better for their health and search for solutions, especially in people with diseases who may be desperate and suffering. However, giving people medical advice without knowing is harmful so the press and journalists need to refrain from jumping to conclusions. This is an example of not considering other factors which may explain the results of disease and microbiomes. It is always important to consider the outside factors before contributing 100% of the results to the independent variables.
For discussing controversy, I think the easiest question to ask would be (5), “could anything else explain these results?” Under this question, you could consider (2), a correlation, (3) if there is a direct mechanism that causes the result, and (4), if it is realistic. With these questions in mind, journalists will have to pause the hype and think critically about the literature before sharing results with the public who are not familiar with the field.
