In W. P. Hanage’s article he discusses 5 questions that are important to take into account when reading scientific literature. The question are being prompted by a new found wave of excitement over the human microbiome.
Can experiments detect differences that matter?
This question is important important because without any detection of real change or change that matters in a persons state there is no reason for it to be used or studied. Some microbial studies did not show any real tangible changes in a person and I think there is a large limitation when no actual change is made. It is important to understand if a study could actually be applied or expand our knowledge of the subject in some way.
Does the study show causation or correlation?
I would argue that this is the most important question that the article poses. In a lot of studies there are only correlations between health changes and differences in the microbiome. It is critical to understand if there is a causal role or if these changes can just happen in a persons microbiome. I also think it is important for studies to indicate that they have only found correlations if a causal relationship hasn’t been established. This is a problem in a lot of interpretations of scientific papers that mistake causation and correlation, which can cause confusion among the general public.
What is the mechanism?
This is an important question because without understanding how a specific mechanism is effected there is know way of knowing causality. This question relates a lot to the last question. Without a mechanism there is almost no way of understanding causality.
How much do experiments reflect reality?
This is another important question because a lot of studies are done in environments where the conditions can be manipulated and might not reflect real world conditions. In a lot of examples of microbiome studies germ free mice are used and I have tended to think of these conditions as sterile and nearly perfect when compared to the real world. Interactions with microbes in the real world tend to be much more complex.
Could anything else explain the results?
Microbiome experiments often have results that do not have a direct causal relationship with outcomes that they are associated with. I think it is important to look at changes in the microbiomes more critically in studies to ensure that other factors do not cofound.
