Tag: computer software

An ‘X’travaganza! Introducing the Marine Mammal Institute’s Center of Drone Excellence (CODEX)

Dr. KC Bierlich, Postdoctoral Scholar, OSU Department of Fisheries, Wildlife, & Conservation Sciences, Geospatial Ecology of Marine Megafauna Lab

Drones are becoming more and more prevalent in marine mammal research, particularly for non-invasively obtaining morphological measurements of cetaceans via photogrammetry to identify important health metrics (see this and this previous blog). For example, the GEMM Lab uses drones for the GRANITE Project to study Pacific Coast Feeding Group (PCFG) gray whales and we have found that PCFG whales are skinnier and morphologically shorter with smaller skulls and flukes compared to the larger Eastern North Pacific (ENP) population. The GEMM Lab has also used drones to document variation in body condition across years and within a season, to diagnose pregnancy, and even measure blowholes.

While drone-based photogrammetry can provide major insight into cetacean ecology, several drone systems and protocols are used across the scientific community in these efforts, and no consistent method or centralized framework is established for quantifying and incorporating measurement uncertainty associated with these different drones. This lack of standardization restricts comparability across datasets, thus hindering our ability to effectively monitor populations and understand the drivers of variation (e.g., pollution, climate change, injury, noise).

We are excited to announce the Marine Mammal Institute’s (MMI) Center of Drone Excellence (CODEX), which focuses on developing analytical methods for using drones to non-invasively monitor marine mammal populations. CODEX is led by GEMM Lab member’s KC Bierlich, Leigh Torres, and Clara Bird and consists of other team members within and outside OSU. We draw from many years of trials, errors, headaches, and effort working with drones to study cetacean ecology in a variety of habitats and conditions on many different species.

Already CODEX has developed several open-source hardware and software tools. We developed, produced, and published LidarBoX (Bierlich et al., 2023), which is a 3D printed enclosure for a LiDAR altimeter system that can be easily attached and swapped between commercially available drones (i.e., DJI Inspire, DJI Mavic, and DJI Phantom) (Figure 1). Having a LidarBoX installed helps researchers obtain altitude readings with greater accuracy, yielding morphological measurements with less uncertainty. Since we developed LidarBoX, we have received over 35 orders to build this unit for other labs in national and international universities.

Figure 1. A ‘LidarBoX’ attached to a DJI Inspire 2. The LidarBoX is a 3D printed enclosure containing a LiDAR altimeter to help obtain more accurate altitude readings.

Additionally, CODEX recently released MorphoMetriX version 2 (v2), an easy-to-use photogrammetry software that provides users with the flexibility to obtain custom morphological measurements of megafauna in imagery with no knowledge of any scripting language (Torres and Bierlich, 2020). CollatriX is a user-friendly software for collating multiple MorphoMetriX outputs into a single dataframe and linking important metadata to photogrammetric measurements, such as altitude measured with a LidarBoX (Bird and Bierlich, 2020). CollatriX also automatically calculates several body condition metrics based on measurements from MorphoMetriX v2. CollatriX v2 is currently in beta-testing and scheduled to be released late Spring 2024. 

Figure 2. An example of a Pygmy blue whale imported into MorphoMetriX v2, open-source photogrammetry software. 

CODEX also recently developed two automated tools to help speed up the laborious manual processing of drone videos for obtaining morphological measurements (Bierlich & Karki et al., in revision). DeteX is a graphical user interface (GUI) that uses a deep learning model for automated detection of cetaceans in drone-based videos. Researchers can input their drone-based videos and DeteX will output frames containing whales at the surface. Users can then select which frames they want to use for measuring individual whales and then input these selected frames into XtraX, which is a GUI that uses a deep learning model to automatically extract body length and body condition measurements of cetaceans (Figure 4). We found automated measurements from XtraX to be similar (within 5%) of manual measurements. Importantly, using DeteX and XtraX takes about 10% of the time it would take to manually process the same videos, demonstrating how these tools greatly speed up obtaining key morphological data while maintaining accuracy, which is critical for effectively monitoring population health.

Figure 3. An example of an automated body length (top) and body condition (bottom) measurement of a gray whale using XtraX (Bierlich & Karki et al., in revision).

CODEX is also in the process of developing Xcertainty, an R package that uses a Bayesian statistical model to quantify and incorporate uncertainty associated with measurements from different drones (see this blog). Xcertainty is based on the Bayesian statistical model developed by Bierlich et al., (2021b; 2021a), which has been utilized by many studies with several different drones to compare body condition and body morphology across individuals and populations  (Bierlich et al., 2022; Torres et al., 2022; Barlow et al., 2023). Rather than a single point-estimate of a length measurement for an individual, Xcertainty produces a distribution of length measurements for an individual so that the length of a whale can be described by the mean of this distribution, and its uncertainty as the the variance or an interval around the mean (Figure 4). These outputs ensure measurements are robust and comparable across different drones because they provide a measure of the uncertainty around each measurement. For instance, a measurement with more uncertainty will have a wider distribution. The uncertainty associated with each measurement can be incorporated into analyses, which is key when detecting important differences or changes in individuals or populations, such as changes in body condition (blog).

Figure 4. An example of a posterior predictive distribution for total length of an individual blue whale produced by the ‘Xcertainty’ R package. The black bars represent the uncertainty around the mean value (the black dot) – the longer black bars represent the 95% highest posterior density (HPD) interval, and the shorter black bars represent the 65% HPD interval. 

CODEX has integrated all these lessons learned, open-source tools, and analytical approaches into a single framework of suggested best practices to help researchers enhance the quality, speed, and accuracy of obtaining important morphological measurements to manage vulnerable populations. These tools and frameworks are designed to be accommodating and accessible to researchers on various budgets and to facilitate cross-lab collaborations. CODEX plans to host workshops to educate and train researchers using drones on how to apply these tools within this framework within their own research practices. Potential future directions for CODEX include developing a system for using drones to drop suction-cup tags on whales and to collect thermal imagery of whales for health assessments. Stay up to date with all the CODEX ‘X’travaganza here: https://mmi.oregonstate.edu/centers-excellence/codex.  

Huge shout out to Suzie Winquist for designing the artwork for CODEX!

References

Barlow, D.R., Bierlich, K.C., Oestreich, W.K., Chiang, G., Durban, J.W., Goldbogen, J.A., Johnston, D.W., Leslie, M.S., Moore, M.J., Ryan, J.P. and Torres, L.G., 2023. Shaped by Their Environment: Variation in Blue Whale Morphology across Three Productive Coastal Ecosystems. Integrative Organismal Biology, [online] 5(1). https://doi.org/10.1093/iob/obad039.

Bierlich, K., Karki, S., Bird, C.N., Fern, A. and Torres, L.G., n.d. Automated body length and condition measurements of whales from drone videos for rapid assessment of population health. Marine Mammal Science.

Bierlich, K.C., Hewitt, J., Bird, C.N., Schick, R.S., Friedlaender, A., Torres, L.G., Dale, J., Goldbogen, J., Read, A.J., Calambokidis, J. and Johnston, D.W., 2021a. Comparing Uncertainty Associated With 1-, 2-, and 3D Aerial Photogrammetry-Based Body Condition Measurements of Baleen Whales. Frontiers in Marine Science, 8. https://doi.org/10.3389/fmars.2021.749943.

Bierlich, K.C., Hewitt, J., Schick, R.S., Pallin, L., Dale, J., Friedlaender, A.S., Christiansen, F., Sprogis, K.R., Dawn, A.H., Bird, C.N., Larsen, G.D., Nichols, R., Shero, M.R., Goldbogen, J., Read, A.J. and Johnston, D.W., 2022. Seasonal gain in body condition of foraging humpback whales along the Western Antarctic Peninsula. Frontiers in Marine Science, 9(1036860), pp.1–16. https://doi.org/10.3389/fmars.2022.1036860.

Bierlich, K.C., Schick, R.S., Hewitt, J., Dale, J., Goldbogen, J.A., Friedlaender, A.S. and Johnston, D.W., 2021b. Bayesian approach for predicting photogrammetric uncertainty in morphometric measurements derived from drones. Marine Ecology Progress Series, 673, pp.193–210. https://doi.org/10.3354/meps13814.

Bird, C. and Bierlich, K.C., 2020. CollatriX: A GUI to collate MorphoMetriX outputs. Journal of Open Source Software, 5(51), pp.2323–2328. https://doi.org/10.21105/joss.02328.

Torres, L.G., Bird, C.N., Rodríguez-González, F., Christiansen, F., Bejder, L., Lemos, L., Urban R, J., Swartz, S., Willoughby, A., Hewitt, J. and Bierlich, K.C., 2022. Range-Wide Comparison of Gray Whale Body Condition Reveals Contrasting Sub-Population Health Characteristics and Vulnerability to Environmental Change. Frontiers in Marine Science, 9(April), pp.1–13. https://doi.org/10.3389/fmars.2022.867258.

Torres, W. and Bierlich, K.C., 2020. MorphoMetriX: a photogrammetric measurement GUI for morphometric analysis of megafauna. Journal of Open Source Software, 5(45), pp.1825–1826. https://doi.org/10.21105/joss.01825.

The Land of Maps and Charts: Geospatial Ecology

By Alexa Kownacki, Ph.D. Student, OSU Department of Fisheries and Wildlife, Geospatial Ecology of Marine Megafauna Lab

I love maps. I love charts. As a random bit of trivia, there is a difference between a map and a chart. A map is a visual representation of land that may include details like topology, whereas a chart refers to nautical information such as water depth, shoreline, tides, and obstructions.

Map of San Diego, CA, USA. (Source: San Diego Metropolitan Transit System)

Chart of San Diego, CA, USA. (Source: NOAA)

I have an intense affinity for visually displaying information. As a child, my dad traveled constantly, from Barrow, Alaska to Istanbul, Turkey. Immediately upon his return, I would grab our standing globe from the dining room and our stack of atlases from the coffee table. I would sit at the kitchen table, enthralled at the stories of his travels. Yet, a story was only great when I could picture it for myself. (I should remind you, this was the early 1990s, GoogleMaps wasn’t a thing.) Our kitchen table transformed into a scene from Master and Commander—except, instead of nautical charts and compasses, we had an atlas the size of an overgrown toddler and salt and pepper shakers to pinpoint locations. I now had the world at my fingertips. My dad would show me the paths he took from our home to his various destinations and tell me about the topography, the demographics, the population, the terrain type—all attribute features that could be included in common-day geographic information systems (GIS).

Uncle Brian showing Alexa where they were on a map of Maui, Hawaii, USA. (Photo: Susan K. circa 1995)

As I got older, the kitchen table slowly began to resemble what I imagine the set from Master and Commander actually looked like; nautical charts, tide tables, and wind predictions were piled high and the salt and pepper shakers were replaced with pencil marks indicating potential routes for us to travel via sailboat. The two of us were in our element. Surrounded by visual and graphical representations of geographic and spatial information: maps. To put my map-attraction this in even more context, this is a scientist who grew up playing “Take-Off”, a board game that was “designed to teach geography” and involved flying your fleet of planes across a Mercator projection-style mapboard. Now, it’s no wonder that I’m a graduate student in a lab that focuses on the geospatial aspects of ecology.

A precocious 3-year-old Alexa, sitting with the airplane pilot asking him a long list of travel-related questions (and taking his captain’s hat). Photo: Susan K.

So why and how did geospatial ecology became a field—and a predominant one at that? It wasn’t that one day a lightbulb went off and a statistician decided to draw out the results. It was a progression, built upon for thousands of years. There are maps dating back to 2300 B.C. on Babylonian clay tablets (The British Museum), and yet, some of the maps we make today require highly sophisticated technology. Geospatial analysis is dynamic. It’s evolving. Today I’m using ArcGIS software to interpolate mass amounts of publicly-available sea surface temperature satellite data from 1981-2015, which I will overlay with a layer of bottlenose dolphin sightings during the same time period for comparison. Tomorrow, there might be a new version of software that allows me to animate these data. Heck, it might already exist and I’m not aware of it. This growth is the beauty of this field. Geospatial ecology is made for us cartophiles (map-lovers) who study the interdependency of biological systems where location and distance between things matters.

Alexa’s grandmother showing Alexa (a very young cartographer) how to color in the lines. Source: Susan K. circa 1994

In a broader context, geospatial ecology communicates our science to all of you. If I posted a bunch of statistical outputs in text or even table form, your eyes might glaze over…and so might mine. But, if I displayed that same underlying data and results on a beautiful map with color-coded symbology, a legend, a compass rose, and a scale bar, you might have this great “ah-ha!” moment. That is my goal. That is what geospatial ecology is to me. It’s a way to SHOW my science, rather than TELL it.

Would you like to see this over and over again…?

A VERY small glimpse into the enormous amount of data that went into this map. This screenshot gave me one point of temperature data for a single location for a single day…Source: Alexa K.

Or see this once…?

Map made in ArcGIS of Coastal common bottlenose dolphin sightings between 1981-1989 with a layer of average sea surface temperatures interpolated across those same years. A picture really is worth a thousand words…or at least a thousand data points…Source: Alexa K.

For many, maps are visually easy to interpret, allowing quick message communication. Yet, there are many different learning styles. From my personal story, I think it’s relatively obvious that I’m, at least partially, a visual learner. When I was in primary school, I would read the directions thoroughly, but only truly absorb the material once the teacher showed me an example. Set up an experiment? Sure, I’ll read the lab report, but I’m going to refer to the diagrams of the set-up constantly. To this day, I always ask for an example. Teach me a new game? Let’s play the first round and then I’ll pick it up. It’s how I learned to sail. My dad described every part of the sailboat in detail and all I heard was words. Then, my dad showed me how to sail, and it came naturally. It’s only as an adult that I know what “that blue line thingy” is called. Geospatial ecology is how I SEE my research. It makes sense to me. And, hopefully, it makes sense to some of you!

Alexa’s dad teaching her how to sail. (Source: Susan K. circa 2000)

Alexa’s first solo sailboat race in Coronado, San Diego, CA. Notice: Alexa’s dad pushing the bow off the dock and the look on Alexa’s face. (Source: Susan K. circa 2000)

Alexa mapping data using ArcGIS in the Oregon State University Library. (Source: Alexa K circa a few minutes prior to posting).

I strongly believe a meaningful career allows you to highlight your passions and personal strengths. For me, that means photography, all things nautical, the great outdoors, wildlife conservation, and maps/charts.  If I converted that into an equation, I think this is a likely result:

Photography + Nautical + Outdoors + Wildlife Conservation + Maps/Charts = Geospatial Ecology of Marine Megafauna

Or, better yet:

? + ⚓ + ? + ? + ? =  GEMM Lab

This lab was my solution all along. As part of my research on common bottlenose dolphins, I work on a small inflatable boat off the coast of California (nautical ✅, outdoors ✅), photograph their dorsal fin (photography ✅), and communicate my data using informative maps that will hopefully bring positive change to the marine environment (maps/charts ✅, wildlife conservation✅). Geospatial ecology allows me to participate in research that I deeply enjoy and hopefully, will make the world a little bit of a better place. Oh, and make maps.

Alexa in the field, putting all those years of sailing and chart-reading to use! (Source: Leila L.)