In their 2014 article, Hanage confronts an issue that has plagued most scientific fields: misinterpretation of scientific literature and the allowance of excitement and novelty to supersede data and its context. As technology becomes more pervasive and media platforms become more accessible, news travels quickly and with it the potential for misinterpretation to the extent of a complete deviation from the initial message. Hanage discusses this potential with a focus on microbiome science. As our knowledge of the human microbiome advances as well as our testing and culturing methods, the potential to answer important questions regarding various human conditions and pathologies has overwhelmed the actual data and what it might mean.
They pose the importance of the following five questions: Can experiments detect differences that matter? Does the study show causation or correlation? What is the mechanism? How much do experiments reflect reality? And could anything else explain the results? (1) These are all important questions which reflect the limitations but also the power of experimentation. Experiments are integral to our growing knowledge, because they afford us the opportunity to remove certain variables from a much larger and complicated system and isolate them in a such a way that we can test them and extract small bits of information given a controlled context. In controlling and limiting the context or environment of the experiment, we can ask specific questions and potentially generate data that supports or rejects those questions. While we can detect differences, those differences do not necessarily reflect what occurs in a natural environment and the data generated is inherently bound by the accuracy and precision of our tools. Experiments are not reflective of reality. They are manipulations of reality, however in isolating little pieces of information, we might build a better perspective of the larger picture. It takes many efforts, approaching the same question from different angles, with different tools and different perspectives to really isolate a meaningful piece of information, whether that be a mechanism or a correlation or a causation. We are always limited by our ignorance. The irony is that we use experimentation to broaden our knowledge but our experimental approach is founded only in what we do understand.
Microbiome science is dynamic; there are essentially an infinite number of players and influences that work simultaneously. We might be able to see different relationships or connections between variables, however there is always the potential that something else could explain the resulting data. The human part of experimentation, whether it lends directly to mistakes and accidents or creativity and efficiency, will always be a variable beyond control. Existence is heterogenous, and it is that heterogeneity which occludes our full comprehension of natural systems – and I believe that the parts that remain unknown are important in our continual search for answers. While correlation and association can be delineated through experimentation, they cannot be construed as causation, because the experimental context is so specific and the natural world is ever-changing. Describing causation comes after extensive, different, redundant studies are completed over a long period of time. While the potential for medical advancements can often pressure claims of correlation, causation like law or theory is a rare finding in science and even when causation can be proven it is typically still described as a spectrum susceptible to different environmental pressures. I struggle to choose which question is most helpful when discussing controversy. They are all important, and in a way similar to why a single experiment does not necessarily prove a hypothesis, but rather a collection of experiments, it is a combination of all of these five questions that will prove most useful in having a constructive discussion about controversy arising over the interpretation of data or experimental results.
1. Hanage W. 2014. Microbiology: Microbiome science needs a healthy dose of scepticism. Nature 512:247-248.
