Finding the right fit: a journey into cetacean distribution models

Solène Derville, Entropie Lab, French National Institute for Sustainable Development (IRD – UMR Entropie), Nouméa, New Caledonia

 Ph.D. student under the co-supervision of Dr. Leigh Torres

Species Distribution Models (SDM), also referred to as ecological niche models, may be defined as “a model that relates species distribution data (occurrence or abundance at known locations) with information on the environmental and/or spatial characteristics of those locations” (Elith & Leathwick, 2009)⁠. In the last couple decades, SDMs have become an indispensable part of the ecologists’ and conservationists’ toolbox. What scientist has not dreamed of being able to summarize a species’ environmental requirements and predict where and when it will occur, all in one tiny statistical model? It sounds like magic… but the short acronym “SDM” is the pretty front window of an intricate and gigantic research field that may extend way beyond the skills of a typical ecologist (even so for a graduate student like myself).

As part of my PhD thesis about the spatial ecology of humpback whales in New Caledonia, South Pacific, I was planning on producing a model to predict their distribution in the region and help spatial planning within the Natural Park of the Coral Sea. An innocent and seemingly perfectly feasible plan for a second year PhD student. To conduct this task, I had at my disposal more than 1,000 sightings recorded during dedicated surveys at sea conducted over 14 years. These numbers seem quite sufficient, considering the rarity of cetaceans and the technical challenges of studying them at sea. And there was more! The NGO Opération Cétacés  also recorded over 600 sightings reported by the general public in the same time period and deployed more than 40 satellite tracking tags to follow individual whale movements. In a field where it is so hard to acquire data, it felt like I had to use it all, though I was not sure how to combine all these types of data, with their respective biases, scales and assumptions.

One important thing about SDM to remember: it is like a cracker section in a US grocery shop, there is sooooo much choice! As I reviewed the possibilities and tested various modeling approaches on my data I realized that this study might be a good opportunity to contribute to the SDM field, by conducting a comparison of various algorithms using cetacean occurrence data from multiple sources. The results of this work was just published  in Diversity and Distributions:

Derville S, Torres LG, Iovan C, Garrigue C. (2018) Finding the right fit: Comparative cetacean distribution models using multiple data sources and statistical approaches. Divers Distrib. 2018;00:1–17. https://doi. org/10.1111/ddi.12782

There are simply too many! Anonymous grocery shops, Corvallis, OR
Credit: Dawn Barlow

If you are a new-comer to the SDM world, and specifically its application to the marine environment, I hope you find this interesting. If you are a seasoned SDM user, I would be very grateful to read your thoughts in the comment section! Feel free to disagree!

So what is the take-home message from this work?

  • There is no such thing as a “best model”; it all depends on what you want your model to be good at (the descriptive vs predictive dichotomy), and what criteria you use to define the quality of your models.

The predictive vs descriptive goal of the model: This is a tricky choice to make, yet it should be clearly identified upfront. Most times, I feel like we want our models to be decently good at both tasks… It is a risky approach to blindly follow the predictions of a complex model without questioning the meaning of the ecological relationships it fitted. On the other hand, conservation applications of models often require the production of predicted maps of species’ probability of presence or habitat suitability.

The criteria for model selection: How could we imagine that the complexity of animal behavior could be summarized in a single metric, such as the famous Akaike Information criterion (AIC) or the Area under the ROC Curve (AUC)? My study, and that of others (e.g. Elith & Graham  H., 2009),⁠ emphasize the importance of looking at multiple aspects of model outputs: raw performance through various evaluation metrics (e.g. see AUCdiff; (Warren & Seifert, 2010)⁠, contribution of the variables to the model, shape of the fitted relationships through Partial Dependence Plots (PDP, Friedman, 2001),⁠ and maps of predicted habitat suitability and associated error. Spread all these lines of evidence in front of you, summarize all the metrics, add a touch of critical ecological thinking to decide on the best approach for your modeling question, and Abracadabra! You end up a bit lost in a pile of folders… But at least you assessed the quality of your work from every angle!

  • Cetacean SDMs often serve a conservation goal. Hence, their capacity to predict to areas / times that were not recorded in the data (which is often scarce) is paramount. This extrapolation performance may be restricted when the model relationships are overfitted, which is when you made your model fit the data so closely that you are unknowingly modeling noise rather than a real trend. Using cross-validation is a good method to prevent overfitting from happening (for a thorough review: Roberts et al., 2017)⁠. Also, my study underlines that certain algorithms inherently have a tendency to overfit. We found that Generalized Additive Models and MAXENT provided a valuable complexity trade-off to promote the best predictive performance, while minimizing overfitting. In the case of GAMs, I would like to point out the excellent documentation that exist on their use (Wood, 2017)⁠, and specifically their application to cetacean spatial ecology (Mannocci, Roberts, Miller, & Halpin, 2017; Miller, Burt, Rexstad, & Thomas, 2013; Redfern et al., 2017).⁠
  • Citizen science is a promising tool to describe cetacean habitat. Indeed, we found that models of habitat suitability based on citizen science largely converged with those based on our research surveys. The main issue encountered when modeling this type of data is the absence of “effort”. Basically, we know where people observed whales, but we do not know where they haven’t… or at least not with the accuracy obtained from research survey data. However, with some information about our citizen scientists and a little deduction, there is actually a lot you can infer about opportunistic data. For instance, in New Caledonia most of the sightings were reported by professional whale-watching operators or by the general public during fishing/diving/boating day trips. Hence, citizen scientists rarely stray far from harbors and spend most of their time in the sheltered waters of the New Caledonian lagoon. This reasoning provides the sort of information that we integrated in our modeling approach to account for spatial sampling bias of citizen science data and improve the model’s predictive performance.

Many more technical aspects of SDM are brushed over in this paper (for detailed and annotated R codes of the modeling approaches, see supplementary information of our paper). There are a few that are not central to the paper, but that I think are worth sharing:

  • Collinearity of predictors: Have you ever found that the significance of your predictors completely changed every time you removed a variable? I have progressively come to discover how unstable a model can be because of predictor collinearity (and the uneasy feeling that comes with it …). My new motto is to ALWAYS check cross-correlation between my predictors, and do it THOROUGHLY. A few aspects that may make a big difference in the estimation of collinearity patterns are to: (1) calculate Pearson vs Spearman coefficients, (2) check correlations between the values recorded at the presence points vs over the whole study area, and (3) assess the correlations between raw environmental variables vs between transformed variables (log-transformed, etc). Though selecting variables with Pearson coefficients < 0.7 is usually a good rule (Dormann et al., 2013), I would worry of anything above 0.5, or at least keep it in mind during model interpretation.
  • Cross-validation: If removing 10% of my dataset greatly impacts the model results, I feel like cross-validation is critical. The concept is based on a simple assumption, if I had sampled a given population/phenomenon/system slightly differently, would I have come to the same conclusion? Cross-validation comes in many different methods, but the basic concept is to run the same model several times (number of times may depend on the size of your data set, hierarchical structure of your data, computation power of your computer, etc.) over different chunks of your data. Model performance metrics (e.g., AUC) and outputs (e.g., partial dependence plots) are than summarized on the many runs, using mean/median and standard deviation/quantiles. It is up to you how to pick these chunks, but before doing this at random I highly recommend reading Roberts et al. (2017).

The evil of the R2: I am probably not the first student to feel like what I have learned in my statistical classes at school is in practice, at best, not very useful, and at worst, dangerously misleading. Of course, I do understand that we must start somewhere, and that learning the basics of inferential statistics is a necessary step to, one day, be able to answer your one research questions. Yet, I feel like I have been carrying the “weight of the R2” for far too long before actually realizing that this metric of model performance (R2 among others) is simply not  enough to trust my results. You might think that your model is robust because among the 1000 alternative models you tested, it is the one with the “best” performance (deviance explained, AIC, you name it), but the model with the best R2 will not always be the most ecologically meaningful one, or the most practical for spatial management perspectives. Overfitting is like a sword of Damocles hanging over you every time you create a statistical model All together, I sometimes trust my supervisor’s expertise and my own judgment more than an R2.

Source: internet

A few good websites/presentations that have helped me through my SDM journey:

General website about spatial analysis (including SDM):

Cool presentation by Adam Smith about SDM:!ecology/sdmShortCourseKState2012/sdmShortCourse_kState.pdf

Handling spatial data in R:

“The magical world of mgcv”, a great presentation by Noam Ross:


Literature cited

Dormann, C. F., Elith, J., Bacher, S., Buchmann, C., Carl, G., Carré, G., … Lautenbach, S. (2013). Collinearity: A review of methods to deal with it and a simulation study evaluating their performance. Ecography, 36(1), 027–046.

Elith, J., & Graham  H., C. (2009). Do they? How do they? WHY do they differ? On finding reasons for differing performances of species distribution models . Ecography, 32(Table 1), 66–77.

Elith, J., & Leathwick, J. R. (2009). Species Distribution Models: Ecological Explanation and Prediction Across Space and Time. Annual Review of Ecology, Evolution, and Systematics, 40(1), 677–697.

Friedman, J. H. (2001). Greedy Function Approximation: A gradient boosting machine. The Annals of Statistics, 29(5), 1189–1232. Retrieved from

Mannocci, L., Roberts, J. J., Miller, D. L., & Halpin, P. N. (2017). Extrapolating cetacean densities to quantitatively assess human impacts on populations in the high seas. Conservation Biology, 31(3), 601–614.

Miller, D. L., Burt, M. L., Rexstad, E. A., & Thomas, L. (2013). Spatial models for distance sampling data: Recent developments and future directions. Methods in Ecology and Evolution, 4(11), 1001–1010.

Redfern, J. V., Moore, T. J., Fiedler, P. C., de Vos, A., Brownell, R. L., Forney, K. A., … Ballance, L. T. (2017). Predicting cetacean distributions in data-poor marine ecosystems. Diversity and Distributions, 23(4), 394–408.

Roberts, D. R., Bahn, V., Ciuti, S., Boyce, M. S., Elith, J., Guillera-Arroita, G., … Dormann, C. F. (2017). Cross-validation strategies for data with temporal, spatial, hierarchical or phylogenetic structure. Ecography, 0, 1–17.

Warren, D. L., & Seifert, S. N. (2010). Ecological niche modeling in Maxent: the importance of model complexity and the performance of model selection criteria. Ecological Applications, 21(2), 335–342.

Wood, S. N. (2017). Generalized additive models: an introduction with R (second edi). CRC press.

Whale mAPP Goes Global!

By Courtney Hann

The Whale mAPP team of Lei Lani Stelle, Melo King of Smallmelo Geographic Information Services, and me (Courtney Hann) has been busy recruiting ocean enthusiasts and applying for grants to fund volunteer-suggested Whale mAPP revisions. As a refresher, Whale mAPP is an Android-based mobile application that can be used by anyone to record marine mammal (whale, dolphins, porpoises, pinniped) sightings around the world. It is easy to sign up as a beta tester on the Whale mAPP website,, and use the mobile app on your personal device to help scientists and conservationists learn more about marine mammals.

What is most spectacular is that the app has truly gone global! We now have over 100 users and are getting a couple new user requests each week! These volunteers have been busy at work this summer, recording thousands of marine mammal sightings during the summer months. Sightings have streamed in from the U.S. West Coast, Hawaiian, Japanese, and Russian waters, the Caribbean, the middle of the Southern Ocean, and the Greenland Sea. You can click on any sighting on the web map to see details about what species were recorded, how many were seen, what their behavior was like, the weather conditions that day, and other notes.

This map shows global marine mammal sightings recorded by Whale mAPP volunteers. The blue whale tale icons represent whale sightings, the pink dolphin icons represent dolphin or porpoise sightings, and the orange seal icon represents a seal, sea lion, or sea otter sighting. Data recorded during each sighting include the location (automatic), the species, the weather and sea conditions, the number of individuals, and a five-star confidence rating (, CEBCO, DeLorme | Melodi King, MS GIS Program – Cohort 21, University of Redlands).
This map shows global marine mammal sightings recorded by Whale mAPP volunteers. The blue whale tale icons represent whale sightings, the pink dolphin icons represent dolphin or porpoise sightings, and the orange seal icon represents a seal, sea lion, or sea otter sighting. Data recorded during each sighting include the location (automatic), the species, the weather and sea conditions, the number of individuals, and a five-star confidence rating (, CEBCO, DeLorme | Melodi King, MS GIS Program – Cohort 21, University of Redlands).

If we zoom into my particular study area of Southeast Alaska, we see that volunteers are still doing a superb job at recording the abundant number of marine mammals in the area.

Zoomed in image of the northeast Pacific, showing marine mammal sightings recorded this summer in Alaska, Southeast Alaska, British Columbia, and Washington (, CEBCO, DeLorme | Melodi King, MS GIS Program – Cohort 21, University of Redlands).
Zoomed in image of the northeast Pacific, showing marine mammal sightings recorded this summer in Alaska, Southeast Alaska, British Columbia, and Washington (, CEBCO, DeLorme | Melodi King, MS GIS Program – Cohort 21, University of Redlands).

Initial funding through the California Coastal Commission Whale Tail grant facilitated the creation of the Whale mAPP project, while current funding from the Hatfield Marine Science Center’s Mamie Markham Research Award that I received this Spring 2015 is funding the top two revisions requested by our users:
(1) Enable the user to edit observations during a trip and after a trip has ended. This revision is huge, was requested by almost every user I spoke to in Southeast Alaska, and was shown very evidently in our survey results.
(2) Add site-specific animal behavior and descriptions to Whale mAPP. For Southeast Alaska, this means adding the famous bubble-net feeding behavior to our list, as well as important descriptions of how to identify, recognize, and understand marine mammal behaviors.
In addition to the above two revisions, a few other revisions will be completed by Smallmelo Geographic Information Services this winter. These revisions include improving world-wide coverage of region-specific species lists, creating tools for validating the quality of the data, enabling data downloads directly from the website (, and including the beta-tested “marine mammal fun facts” into the global application.

All of these incredible accomplishments and progress toward a successful, educational, fun, and data-generating marine mammal citizen science project could not have happened without Dr. Lei Lani’s open mind toward incorporating more people, including me, into her Whale mAPP project. Whale mAPP represents a new age of scientists, one of collaboration across disciplines (ecology, statistics, coding, education, and more!), and one that over-steps previous boundaries to re-define science. I hope that my participation with Whale mAPP and future citizen science projects will inspire individuals to know and feel that they can be scientists and that we can inspire the world to work together for the common good of our oceans.