An ‘X’travaganza! Introducing the Marine Mammal Institute’s Center of Drone Excellence (CODEX)

Dr. KC Bierlich, Postdoctoral Scholar, OSU Department of Fisheries, Wildlife, & Conservation Sciences, Geospatial Ecology of Marine Megafauna Lab

Drones are becoming more and more prevalent in marine mammal research, particularly for non-invasively obtaining morphological measurements of cetaceans via photogrammetry to identify important health metrics (see this and this previous blog). For example, the GEMM Lab uses drones for the GRANITE Project to study Pacific Coast Feeding Group (PCFG) gray whales and we have found that PCFG whales are skinnier and morphologically shorter with smaller skulls and flukes compared to the larger Eastern North Pacific (ENP) population. The GEMM Lab has also used drones to document variation in body condition across years and within a season, to diagnose pregnancy, and even measure blowholes.

While drone-based photogrammetry can provide major insight into cetacean ecology, several drone systems and protocols are used across the scientific community in these efforts, and no consistent method or centralized framework is established for quantifying and incorporating measurement uncertainty associated with these different drones. This lack of standardization restricts comparability across datasets, thus hindering our ability to effectively monitor populations and understand the drivers of variation (e.g., pollution, climate change, injury, noise).

We are excited to announce the Marine Mammal Institute’s (MMI) Center of Drone Excellence (CODEX), which focuses on developing analytical methods for using drones to non-invasively monitor marine mammal populations. CODEX is led by GEMM Lab member’s KC Bierlich, Leigh Torres, and Clara Bird and consists of other team members within and outside OSU. We draw from many years of trials, errors, headaches, and effort working with drones to study cetacean ecology in a variety of habitats and conditions on many different species.

Already CODEX has developed several open-source hardware and software tools. We developed, produced, and published LidarBoX (Bierlich et al., 2023), which is a 3D printed enclosure for a LiDAR altimeter system that can be easily attached and swapped between commercially available drones (i.e., DJI Inspire, DJI Mavic, and DJI Phantom) (Figure 1). Having a LidarBoX installed helps researchers obtain altitude readings with greater accuracy, yielding morphological measurements with less uncertainty. Since we developed LidarBoX, we have received over 35 orders to build this unit for other labs in national and international universities.

Figure 1. A ‘LidarBoX’ attached to a DJI Inspire 2. The LidarBoX is a 3D printed enclosure containing a LiDAR altimeter to help obtain more accurate altitude readings.

Additionally, CODEX recently released MorphoMetriX version 2 (v2), an easy-to-use photogrammetry software that provides users with the flexibility to obtain custom morphological measurements of megafauna in imagery with no knowledge of any scripting language (Torres and Bierlich, 2020). CollatriX is a user-friendly software for collating multiple MorphoMetriX outputs into a single dataframe and linking important metadata to photogrammetric measurements, such as altitude measured with a LidarBoX (Bird and Bierlich, 2020). CollatriX also automatically calculates several body condition metrics based on measurements from MorphoMetriX v2. CollatriX v2 is currently in beta-testing and scheduled to be released late Spring 2024. 

Figure 2. An example of a Pygmy blue whale imported into MorphoMetriX v2, open-source photogrammetry software. 

CODEX also recently developed two automated tools to help speed up the laborious manual processing of drone videos for obtaining morphological measurements (Bierlich & Karki et al., in revision). DeteX is a graphical user interface (GUI) that uses a deep learning model for automated detection of cetaceans in drone-based videos. Researchers can input their drone-based videos and DeteX will output frames containing whales at the surface. Users can then select which frames they want to use for measuring individual whales and then input these selected frames into XtraX, which is a GUI that uses a deep learning model to automatically extract body length and body condition measurements of cetaceans (Figure 4). We found automated measurements from XtraX to be similar (within 5%) of manual measurements. Importantly, using DeteX and XtraX takes about 10% of the time it would take to manually process the same videos, demonstrating how these tools greatly speed up obtaining key morphological data while maintaining accuracy, which is critical for effectively monitoring population health.

Figure 3. An example of an automated body length (top) and body condition (bottom) measurement of a gray whale using XtraX (Bierlich & Karki et al., in revision).

CODEX is also in the process of developing Xcertainty, an R package that uses a Bayesian statistical model to quantify and incorporate uncertainty associated with measurements from different drones (see this blog). Xcertainty is based on the Bayesian statistical model developed by Bierlich et al., (2021b; 2021a), which has been utilized by many studies with several different drones to compare body condition and body morphology across individuals and populations  (Bierlich et al., 2022; Torres et al., 2022; Barlow et al., 2023). Rather than a single point-estimate of a length measurement for an individual, Xcertainty produces a distribution of length measurements for an individual so that the length of a whale can be described by the mean of this distribution, and its uncertainty as the the variance or an interval around the mean (Figure 4). These outputs ensure measurements are robust and comparable across different drones because they provide a measure of the uncertainty around each measurement. For instance, a measurement with more uncertainty will have a wider distribution. The uncertainty associated with each measurement can be incorporated into analyses, which is key when detecting important differences or changes in individuals or populations, such as changes in body condition (blog).

Figure 4. An example of a posterior predictive distribution for total length of an individual blue whale produced by the ‘Xcertainty’ R package. The black bars represent the uncertainty around the mean value (the black dot) – the longer black bars represent the 95% highest posterior density (HPD) interval, and the shorter black bars represent the 65% HPD interval. 

CODEX has integrated all these lessons learned, open-source tools, and analytical approaches into a single framework of suggested best practices to help researchers enhance the quality, speed, and accuracy of obtaining important morphological measurements to manage vulnerable populations. These tools and frameworks are designed to be accommodating and accessible to researchers on various budgets and to facilitate cross-lab collaborations. CODEX plans to host workshops to educate and train researchers using drones on how to apply these tools within this framework within their own research practices. Potential future directions for CODEX include developing a system for using drones to drop suction-cup tags on whales and to collect thermal imagery of whales for health assessments. Stay up to date with all the CODEX ‘X’travaganza here: https://mmi.oregonstate.edu/centers-excellence/codex.  

Huge shout out to Suzie Winquist for designing the artwork for CODEX!

References

Barlow, D.R., Bierlich, K.C., Oestreich, W.K., Chiang, G., Durban, J.W., Goldbogen, J.A., Johnston, D.W., Leslie, M.S., Moore, M.J., Ryan, J.P. and Torres, L.G., 2023. Shaped by Their Environment: Variation in Blue Whale Morphology across Three Productive Coastal Ecosystems. Integrative Organismal Biology, [online] 5(1). https://doi.org/10.1093/iob/obad039.

Bierlich, K., Karki, S., Bird, C.N., Fern, A. and Torres, L.G., n.d. Automated body length and condition measurements of whales from drone videos for rapid assessment of population health. Marine Mammal Science.

Bierlich, K.C., Hewitt, J., Bird, C.N., Schick, R.S., Friedlaender, A., Torres, L.G., Dale, J., Goldbogen, J., Read, A.J., Calambokidis, J. and Johnston, D.W., 2021a. Comparing Uncertainty Associated With 1-, 2-, and 3D Aerial Photogrammetry-Based Body Condition Measurements of Baleen Whales. Frontiers in Marine Science, 8. https://doi.org/10.3389/fmars.2021.749943.

Bierlich, K.C., Hewitt, J., Schick, R.S., Pallin, L., Dale, J., Friedlaender, A.S., Christiansen, F., Sprogis, K.R., Dawn, A.H., Bird, C.N., Larsen, G.D., Nichols, R., Shero, M.R., Goldbogen, J., Read, A.J. and Johnston, D.W., 2022. Seasonal gain in body condition of foraging humpback whales along the Western Antarctic Peninsula. Frontiers in Marine Science, 9(1036860), pp.1–16. https://doi.org/10.3389/fmars.2022.1036860.

Bierlich, K.C., Schick, R.S., Hewitt, J., Dale, J., Goldbogen, J.A., Friedlaender, A.S. and Johnston, D.W., 2021b. Bayesian approach for predicting photogrammetric uncertainty in morphometric measurements derived from drones. Marine Ecology Progress Series, 673, pp.193–210. https://doi.org/10.3354/meps13814.

Bird, C. and Bierlich, K.C., 2020. CollatriX: A GUI to collate MorphoMetriX outputs. Journal of Open Source Software, 5(51), pp.2323–2328. https://doi.org/10.21105/joss.02328.

Torres, L.G., Bird, C.N., Rodríguez-González, F., Christiansen, F., Bejder, L., Lemos, L., Urban R, J., Swartz, S., Willoughby, A., Hewitt, J. and Bierlich, K.C., 2022. Range-Wide Comparison of Gray Whale Body Condition Reveals Contrasting Sub-Population Health Characteristics and Vulnerability to Environmental Change. Frontiers in Marine Science, 9(April), pp.1–13. https://doi.org/10.3389/fmars.2022.867258.

Torres, W. and Bierlich, K.C., 2020. MorphoMetriX: a photogrammetric measurement GUI for morphometric analysis of megafauna. Journal of Open Source Software, 5(45), pp.1825–1826. https://doi.org/10.21105/joss.01825.

Who, where, when: Estimating individual space use patterns of PCFG gray whales

By Lisa Hildebrand, PhD candidate, OSU Department of Fisheries, Wildlife, & Conservation Sciences, Geospatial Ecology of Marine Megafauna Lab

Understanding how baleen whales are affected by human activity is a central goal for many research projects in the GEMM Lab. The overarching goal of the GRANITE (Gray whale Response to Ambient Noise Informed by Technology and Ecology) project is to quantify baleen whale physiological response to different stressors (e.g., boat presence and noise) and model the subsequent impacts of these stressors on the population. We will achieve this goal by implementing our long-term, replicate dataset of Pacific Coast Feeding Group (PCFG) gray whales into a framework called population consequences of disturbance (PCoD). I will not go into the details of PCoD in this blog (but I wrote a post a few years ago that you can revisit). Instead, I will explain the approach I am taking to assess where and when individual whales spend time in our study area, which will form an essential component of PCoD and be one of the chapters of my PhD dissertation.

Individuals in a population are unlikely to be exposed to a stressor in a uniform way because they make decisions differently based on intrinsic (e.g., sex, age, reproductive status) and extrinsic (e.g., environment, prey, predators) factors (Erlinge & Sandell 1986). For example, a foraging female gray whale who is still nursing a calf will need to consider factors that are different to ones that an adult single male might need to consider when choosing a location to feed. These differences in decision-making exist across the whole population, which makes it important to understand where individuals are spending time and how they overlap with stressors in space and time before trying to quantify the impacts of stressors on the population as a whole (Pirotta et al. 2018). I am currently working on an analysis that will determine an individual’s exposure to a number of stressors based on their space use patterns. 

We can monitor space use patterns of individuals in a population through time using spatial capture-recapture techniques. As the name implies, a spatial capture-recapture technique involves capturing an individual in a marked location during a sampling period, releasing it back into the population, and then (hopefully) re-capturing it during another sampling period in the future, at either the same or a different location. With enough repeat sampling events, the method should build spatial capture histories of individuals through time to better understand an individual’s space use patterns (Borchers & Efford 2008). While the use of the word capture implies that the animal is being physically caught, this is not necessarily the case. Individuals can be “captured” in a number of non-invasive ways, including by being photographed, which is how we “capture” individual PCFG gray whales. These capture-recapture methods were first pioneered in terrestrial systems, where camera traps (i.e., cameras that take photos or videos when a motion sensor is triggered) are set up in a systematic grid across a study area (Figure 1; Royle et al. 2009, Gray 2018). Placing the cameras in a grid system ensures that there is an equal distribution of cameras throughout the study area, which means that an animal theoretically has a uniform chance of being captured. However, because we know that individuals within a population make space use decisions differently, we assume that individuals will distribute themselves differently across a landscape, which will manifest as individuals having different centers of their spatial activity. The probability of capturing an individual is highest when a camera trap is at that individual’s activity center, and the cameras furthest away from the individual’s activity center will have the lowest probability of capturing that individual (Efford 2004). By using this principle of probability, the data generated from spatial capture-recapture field methods can be modelled to estimate the activity centers and ranges for all individuals in a population. The overlap of an individual’s activity center and range can then be compared to the spatiotemporal distribution of stressors that an individual may be exposed to, allowing us to determine whether and how an individual has been exposed to each stressor. 

Figure 1. Example of camera trap grid in a study area. Figure taken from Gray (2018).

While capture-recapture methods were first developed in terrestrial systems, they have been adapted for application to marine populations, which is what I am doing for our GRANITE dataset of PCFG gray whales. Together with a team of committee members and GRANITE collaborators, I am developing a Bayesian spatial capture-recapture model to estimate individual space use patterns. In order to mimic the camera trap grid system, we have divided our central Oregon coast study area into latitudinal bins that are approximately 1 km long. Unfortunately, we do not have motion sensor activated cameras that automatically take photographs of gray whales in each of these latitudinal bins. Instead, we have eight years of boat-based survey effort with whale encounters where we collect photographs of many individual whales. However, as you now know, being able to calculate the probability of detection is important for estimating an individual’s activity center and range. Therefore, we calculated our spatial survey effort per latitudinal bin in each study year to account for our probability of detecting whales (i.e., the area of ocean in km2 that we surveyed). Next, we tallied up the number of times we observed every individual PCFG whale in each of those latitudinal bins per year, thus creating individual spatial capture histories for the population. Finally, using just those two data sets (the individual whale capture histories and our survey effort), we can build models to test a number of different hypotheses about individual gray whale space use patterns. There are many hypotheses that I want to test (and therefore many models that I need to run), with increasing complexity, but I will explain one here.

Over eight years of field work for the GRANITE project, consisting of over 40,000 km2 of ocean surveyed with 2,169 sightings of gray whales, our observations lead us to hypothesize that there are two broad space use strategies that whales use to optimize how they find enough prey to meet their energetic needs. For the moment, we are calling these strategies ‘home-body’ and ‘roamer’. As the name implies, a home-body is an individual that stays in a relatively small area and searches for food in this area consistently through time. A roamer, on the other hand, is an individual that travels and searches over a greater spatial area to find good pockets of food and does not generally tend to stay in just one place. In other words, we except a home-body to have a consistent activity center through time and a small activity range, while a roamer will have a much larger activity range and its activity center may vary more throughout the years (Figure 2). 

Figure 2. Schematic representing one of the hypotheses we will be testing with our Bayesian spatial capture-recapture models. The schematic shows the activity centers (the circles) and activity ranges (vertical lines attached to the circles) of two individuals (green and orange) across three years in our central Oregon study area. The green individual represents our hypothesized idea of a home-body, whereas the orange individuals represents our idea of a roamer.

While this hypothesis sounds straightforward, there are a lot of decisions that I need to make in the Bayesian modeling process that can ultimately impact the results. For example, do all home-bodies in a population have the same size activity range or can the size vary between different home-bodies? If it can vary, by how much can it vary? These same questions apply for the roamers too. I have a long list of questions just like these, which means a lot of decision-making on my part, and that long list of hypotheses I previously mentioned. Luckily, I have a fantastic team made up of Leigh, committee members, and GRANITE collaborators that are guiding me through this process. In just a few more months, I hope to reveal how PCFG individuals distribute themselves in space and time throughout our central Oregon study area, and hence describe their exposure to different stressors. Stay tuned! 

Did you enjoy this blog? Want to learn more about marine life, research, and conservation? Subscribe to our blog and get a weekly message when we post a new blog. Just add your name and email into the subscribe box below.

Loading

References

Borchers DL, Efford MG (2008) Spatially explicit maximum likelihood methods for capture-recapture studies. Biometrics 64:377-385.

Efford M (2004) Density estimation in live-trapping studies. Oikos 106:598-610.

Erlinge S, Sandell M (1986) Seasonal changes in the social organization of male stoats, Mustela erminea: An effect of shifts between two decisive resources. Oikos 47:57-62.

Gray TNE (2018) Monitoring tropical forest ungulates using camera-trap data. Journal of Zoology 305:173-179.

Pirotta E, Booth CG, Costa DP, Fleishman E, Kraus SD, and others (2018) Understanding the population consequences of disturbance. Ecology and Evolution 8(19):9934–9946.Royle J, Nichols J, Karanth KU, Gopalaswamy AM (2009) A hierarchical model for estimating density in camera-trap studies. Journal of Applied Ecology 46:118-127.