Month: September 2022

Decisions, decisions: New GEMM Lab publication reveals trade-offs in prey quantity and quality in gray whale foraging

By Lisa Hildebrand, PhD student, OSU Department of Fisheries, Wildlife, & Conservation Sciences, Geospatial Ecology of Marine Megafauna Lab

Obtaining enough food is crucial for predators to ensure adequate energy gain for maintenance of vital functions and support for energetically costly life history events (e.g., reproduction). Foraging involves decisions at every step of the process, including prey selection, capture, and consumption, all of which should be as efficient as possible. Making poor foraging decisions can have long-term repercussions on reproductive success and population dynamics (Harris et al. 2007, 2008, Grémillet et al. 2008), and for marine predators that rely on prey that is spatially and temporally dynamic and notoriously patchy (Hyrenbach et al. 2000), these decisions can be especially challenging. Prey abundance and density are frequently used as predictors of marine predator distribution, movement, and foraging effort, with predators often selecting highly abundant or dense prey patches (e.g., Goldbogen et al. 2011, Torres et al. 2020). However, there is increased recognition that prey quality is also an important factor to consider when assessing a predator’s ecology and habitat use (Spitz et al. 2012), and marine predators do show a preference for higher quality prey items (e.g., Haug et al. 2002, Cade et al. 2022). Moreover, negative impacts of low-quality prey on the health and breeding success of some marine mammals (Rosen & Trites 2000, Trites & Donnelly 2003) have been documented. Therefore, examining multiple prey metrics, such as prey quantity and quality, in predator ecology studies is critical.

Figure 1. Site map of the Port Orford TOPAZ/JASPER integrated projects. Blue squares represent the location of the 12 sampling stations within the 2 study sites (site boundaries demarcated with black lines). Brown dot represents the cliff-top observation site where theodolite tracking occurred.

Our integrated TOPAZ/JASPER projects in Port Orford do just this! We collect both prey quantity and quality data from a tandem research kayak, while we track Pacific Coast Feeding Group (PCFG) gray whales from shore. The prey and whale sampling overlap spatially (and often temporally within the same day). This kind of concurrent predator-prey sampling at similar scales is often logistically challenging to achieve, yet because PCFG gray whales have an affinity for nearshore, coastal habitats, it is something we have been able to achieve in Port Orford. Since 2016, a field team comprised of graduate, undergraduate, and high school students has collected data during the month of August to investigate gray whale foraging decisions relative to prey. Every day, a kayak team collects GoPro videos (to assess relative prey abundance; AKA: quantity) and zooplankton samples using a tow net (to assess prey community composition; AKA: quality through caloric content of different species) (Figure 1). At the same time, a cliff team surveys for gray whales from shore and tracks them using a theodolite, which provides us with tracklines of individual whales; We categorize each location of a whale into three broad behavior states (feeding, searching, transiting) based on movement patterns. Over the years, the various students who have participated in the TOPAZ/JASPER projects have written many blog posts, which I encourage you to read here (particularly to get more detailed information about the field methods). 

Figure 2. An example daily layer of relative prey abundance (increasing color darkness corresponds with increasing abundance) in one study site with a whale theodolite trackline recorded on the same day overlaid and color-coded by behavioral state.

Several years of data are needed to conduct a robust analysis for our ecological questions about prey choice, but after seven years, we finally had the data and I am excited to share the results, which are due to the many years of hard work from many students! Our recent paper in Marine Ecology Progress Series aimed to determine whether PCFG gray whale foraging decisions are driven by prey quantity (abundance) or quality (caloric content of species) at a scale of 20 m (which is slightly less than 2 adult gray whale body lengths). In this study, we built upon results from my previous Master’s publication, which revealed that there are significant differences in the caloric content between the six common nearshore zooplankton prey species that PCFG gray whales feed on (Hildebrand et al. 2021). Therefore, in this study we addressed the hypothesis that individual whales will select areas where the prey community is dominated by the mysid shrimp Neomysis rayii, since it is significantly higher in caloric content than the other two prey species we identified, Holmesimysis sculpta (a medium quality mysid shrimp species) and Atylus tridens (a low quality amphipod species) (Hildebrand et al. 2021). We used spatial statistics and model to make daily maps of prey abundance and quality that we compared to our whale tracks and behavior from the same day. Please read our paper for the details on our novel methods that produced a dizzying amount of prey layers, which allowed us to tease apart whether gray whales target prey quantity, quality, or a mixture of both when they forage. 

Figure 3. Figure shows the probability of gray whale foraging relative to prey abundance (color-coded by prey species). Dark grey vertical line represents the mean threshold for the H. sculpta curves (12.0); light grey vertical lines: minimum (7.2) and maximum (15.3) thresholds for the H. sculpta curves. Inflection points could not be calculated for the N. rayii curves

So, what did we find? The models proved our hypothesis wrong: foraging probability was significantly correlated with the quantity and quality of the mysid H. sculpta, which has significantly lower calories than N. rayii. This result puzzled us, until we started looking at the overall quantity of these two prey types in the study area and realized that the amount of calorically-rich N. rayii never reached a threshold where it was beneficial for gray whales to forage. But, there was a lot of H. sculpta, which likely made for an energetic gain for the whales despite not being as calorically rich as N. rayii. We determined a threshold of H. sculpta relative abundance that is required to initiate the gray whale foraging behavior, and the abundance of N. rayii never came close to this level (Figure 3). Despite not having the highest quality, H. sculpta did have the highest abundance and showed a significant positive relationship with foraging behavior, unlike the other prey items. Interestingly, whales never selected areas dominated by the low-calorie species A. tridens. These results demonstrate trade-off choices by whales for this abundant, medium-quality prey.

To our knowledge, individual baleen whale foraging decisions relative to available prey quantity and quality have not been addressed previously at this very fine-scale. Interestingly, this trade-off between prey quantity and quality has also been detected in humpback whales foraging in Antarctica at depths deeper than where the densest krill patches occur; while the whales are exploiting less dense krill patches, these krill composed of larger, gravid, higher-quality krill (Cade et al. 2022). While it is unclear how baleen whales differentiate between prey species or reproductive stages, several mechanisms have been suggested, including visual and auditory identification (Torres 2017). We assume here that gray whales, and other baleen whale species, can differentiate between prey species. Thus, our results showcase the importance of knowing the quality (such as caloric content) of prey items available to predators to understand their foraging ecology (Spitz et al. 2012). 

References

Cade DE, Kahane-Rapport SR, Wallis B, Goldbogen JA, Friedlaender AS (2022) Evidence for size-selective pre- dation by Antarctic humpback whales. Front Mar Sci 9:747788

Goldbogen JA, Calambokidis J, Oleson E, Potvin J, Pyenson ND, Schorr G, Shadwick RE (2011) Mechanics, hydrody- namics and energetics of blue whale lunge feeding: effi- ciency dependence on krill density. J Exp Biol 214:131−146

Grémillet D, Pichegru L, Kuntz G, Woakes AG, Wilkinson S, Crawford RJM, Ryan PG (2008) A junk-food hypothesis for gannets feeding on fishery waste. Proc R Soc B 275: 1149−1156

Harris MP, Beare D, Toresen R, Nøttestad L, and others (2007) A major increase in snake pipefish (Entelurus aequoreus) in northern European seas since 2003: poten- tial implications for seabird breeding success. Mar Biol 151:973−983

Harris MP, Newell M, Daunt F, Speakman JR, Wanless S (2008) Snake pipefish Entelurus aequoreus are poor food for seabirds. Ibis 150:413−415

Haug T, Lindstrøm U, Nilssen KT (2002) Variations in minke whale (Balaenoptera acutorostrata) diet and body condi- tion in response to ecosystem changes in the Barents Sea. Sarsia 87:409−422

Hildebrand L, Bernard KS, Torres LG (2021) Do gray whales count calories? Comparing energetic values of gray whale prey across two different feeding grounds in the eastern North Pacific. Front Mar Sci 8:1008

Hyrenbach KD, Forney KA, Dayton PK (2000) Marine pro- tected areas and ocean basin management. Aquat Con- serv 10:437−458

Rosen DAS, Trites AW (2000) Pollock and the decline of Steller sea lions: testing the junk-food hypothesis. Can J Zool 78:1243−1250

Spitz J, Trites AW, Becquet V, Brind’Amour A, Cherel Y, Galois R, Ridoux V (2012) Cost of living dictates what whales, dolphins and porpoises eat: the importance of prey quality on predator foraging strategies. PLOS ONE 7:e50096

Torres LG, Barlow DR, Chandler TE, Burnett JD (2020) Insight into the kinematics of blue whale surface forag- ing through drone observations and prey data. PeerJ 8: e8906

Torres LG (2017) A sense of scale: foraging cetaceans’ use of scale-dependent multimodal sensory systems. Mar Mamm Sci 33:1170−1193

Trites AW, Donnelly CP (2003) The decline of Steller sea lions Eumetopias jubatus in Alaska: a review of the nutri- tional stress hypothesis. Mammal Rev 33:3−28

Keeping up with the HALO project: Recovering Rockhopper acoustic recording units and eavesdropping on Northern right whale dolphins

Marissa Garcia, PhD Student, Cornell University, Department of Natural Resources and the Environment, K. Lisa Yang Center for Conservation Bioacoustics

It was a June morning on the Pacific Ocean, and the R/V Pacific Storm had come to a halt on its journey back to shore. The night before, the Holistic Assessment of Living marine resources off Oregon (HALO) project team had disembarked from Newport and began the long transit to NH 65, a site 65 nautical miles offshore along the Newport Hydrographic line (NH line). Ever since the 1960s, researchers have been conducting oceanographic studies along the NH line; the HALO project seeks to add the biological dimension to these historical data collections.

We were on a mission to recover our first set of Rockhoppers that we had deployed in October 2021, just nine months earlier. The Rockhopper is an underwater passive acoustic recording unit developed by K. Lisa Yang Center for Conservation Bioacoustics at Cornell University. Earlier versions of underwater recorders were optimized to record baleen whales. By contrast, the Rockhopper is designed to record both baleen whales and dolphins on longer and deeper deployments, making it apt for research endeavors such as the HALO project. Three units, deployed at NH 25, 45, and 65, continuously recorded the soundscape of the Oregon waters for six months. In June, we were headed out to sea to recover these three units, collect the acoustic data, and deploy three new units.

Figure 1: The HALO project routinely surveys the trackline spanning between NH 25 and NH 65 on the NH line. Credit: Leigh Torres.

With the ship paused, our first task was to recover the Rockhopper we had deployed at NH 65. This Rockhopper deployment at NH 65 was our deepest successful deployment to date, moored at nearly 3,000 m.

So, how does one recover an underwater recording unit that is nearly 3,000 m below the surface? When the Rockhopper was deployed, it was anchored to the seafloor with a 60 kg cast iron anchor. It seems improbable that an underwater recording unit — anchored by such heavy weights — can eventually rise to the surface, but this capability is made possible through a piece of attached equipment called the acoustic release. By sending a signal of a numbered code from a box on the boat deck through the water column to the Rockhopper, the bottom of the acoustic release will begin to spin and detach from the weights. The weights are then left on the seafloor, as the Rockhopper slowly rises to the surface, now unhindered by the weights. Since these weights are composed of iron, they will naturally erode, without additional pollution contributed to the ecosystem. At NH 65, it took approximately an hour for the Rockhopper to reach the surface.

Figure 2: A diagram of the Rockhopper mooring. Of particular importance to this blog post is the acoustic release (Edgtech PORT MFE release) and the 60 kg anchor (Source: Klinck et al., 2020).

The next challenge is finding the Rockhopper bobbing amongst the waves in the vast ocean — much like searching for a needle in a haystack. The color of the Rockhopper helps aid in this quest. It’s imperative anyone out on the boat deck wears a life jacket; if someone goes overboard while wearing a life-jacket, on-board passengers can more easily spot a bright orange spot in an otherwise blue-green ocean with white caps. The design of the Rockhopper functions similarly; the unit is contained in a bright orange hard hat, helping researchers on-board to more easily spot the device, especially in an ocean often characterized by high sea state.

We also use a Yagi antenna to listen for the VHF (Very High Frequency) signal of the recovery gear, a signal the Rockhopper emits once it’s surfaced above the waterline. Pointing the antenna toward the ocean, we can detect the signal, which will become stronger when we point antenna in the direction of the Rockhopper; once we hear that strong signal, we can recommend to the boat captain to start moving the vessel in that direction.

Figure 3: Derek Jaskula, a member of the field operations team at the K. Lisa Yang Center for Conservation Bioacoustics, points the Yagi antenna to detect the signal from the surfaced Rockhopper. Credit: Marissa Garcia.

At that point, all eyes are on the water, binoculars scanning the horizon for the orange. All ears are eager for the exciting news: “I see the Rockhopper!”

Once that announcement is made, the vessel carefully inches toward the Rockhopper until it is just next to the vessel’s side. Using a hook, the Rockhopper is pulled upward and back onto the deck.

What we weren’t expecting, however, during this recovery was to have our boat surrounded by two dolphin species: Pacific white-sided dolphins (Lagenorhynchus obliquidens) and Northern right whale dolphins (Lissodelphis borealis).

One HALO team member shouted, “I see Northern right whale dolphins!”

Charged with excitement, I quickly climbed up the crow’s nest to get a birds-eye look at the ocean bubbling around us with surfacing dolphins. Surely enough, I spotted the characteristic stripe of the Pacific-white sided dolphins zooming beneath the surface, in streaks of white. But what I was even more eager to see were the Northern right whale dolphins, flipping themselves out of the water, unveiling their bright white undersides. Because they lack dorsal fins, we on-board colloquially refer to Northern right whale dolphins as “sea slugs” to describe their appearance as they surface.

Figure 4: The Northern right whale dolphin (Lissodelphis borealis) surfaces during a HALO cruise. Source: HALO Project Team Member. Permit: NOAA/NMFS permit #21678.

In my analysis of the HALO project data for my PhD, I am interested in using acoustics to describe how the distribution of dolphins and toothed whales in Oregon waters varies across space and time. One species I am especially fascinated to study in-depth is the Northern right whale dolphin. To my knowledge, only three papers to date have attempted to describe their acoustics — two of which were published in the 1970s, and the most recent of which was published fifteen years ago (Fish & Turl, 1976; Leatherwood & Walker, 1979; Rankin et al., 2007).

Leatherwood & Walker (1979) proposed that Northern right whale dolphins produced two categories of whistles: a high frequency whistle that turned into burst-pulse vocalizations, and low frequency whistles. However, Rankin et al. (2007) proposed that Northern right whale dolphins may not actually produce whistles, based on two lines of evidence. First, Rankin et al. (2007) combined visual and acoustic survey, and all vocalizations recorded were localized via beamforming methods to verify that recorded vocalizations were produced by the visually observed dolphins. The visual surveying component is key to validating the vocalizations of the species, which also hints that the HALO project’s multi-surveying approach (acoustic and visual) could help arrive at similar results. Second, the Rankin et al. (2007) explored the taxonomy of the Northern right whale dolphin to verify which vocalizations the species is likely to produce based on the vocal repertoire of its close relatives. The right whale dolphin is closely related to dolphins in the genus Lagenorhynchus — which includes white-sided dolphins — and Cephalorhynchus — which includes Hector’s dolphin. The vocal repertoire of these relatives don’t produce whistles, and instead predominantly produced pulsed sounds or clicks (Dawson, 1991; Herman & Tavolga, 1980). Northern right whale dolphins primarily produce echolocation clicks trains and burst-pulses. Although Rankin et al. (2007) claims that the Northern right whale dolphin does not produce whistles, stereotyped burst-pulse series may be unique to individuals, just as dolphin species use stereotyped signature whistles, or they may be relationally shared just as discrete calls of killer whales are.

Figure 5: The Northern right whale dolphin (Lissodelphis borealis) produces burst-pulses. There exists variation in series of burst-pulses. The units marked by (a) and (b) ultimately get replaced by the unit marked by (c). (Source: Rankin et al., 2007).

We have just finished processing the first round of acoustic data for the HALO project, and it is ready now for analysis. Already previewing an hour of data on the Rockhopper by NH 25, we identified potential Northern right whale dolphin recordings . So far, we have only visually observed Northern right whale dolphins nearby Rockhopper units placed at sites NH 65 and NH 45, so it was surprising to acoustically detect this species on the most inshore unit at NH 25. I look forward to demystifying the mystery of Northern right whale dolphin vocalizations as our research on the HALO project continues!

Figure 6: Potential Northern right whale dolphin vocalizations recorded at the Rockhopper deployed at NH 25.

Did you enjoy this blog? Want to learn more about marine life, research and conservation? Subscribe to our blog and get weekly updates and more! Just add your name into the subscribe box below! 

References

Dawson, S. (1991). Clicks and Communication: The Behavioural and Social Contexts of Hector’s Dolphin Vocalizations. Ethology, 88(4), 265–276. https://doi.org/10.1111/j.1439-0310.1991.tb00281.x

Fish, J. F. & Turl, C. W. (1976). Acoustic Source Levels of Four Species of Small Whales.

Herman, L. M., and Tavolga, W. N. (1980). “The communication systems of cetaceans,” in Cetacean behavior: Mechanisms and functions, edited by L. M. Herman (Wiley, New York), 149–209.

Klinck, H., Winiarski, D., Mack, R. C., Tessaglia-Hymes, C. T., Ponirakis, D. W., Dugan, P. J., Jones, C., & Matsumoto, H. (2020). The Rockhopper: a compact and extensible marine autonomous passive acoustic recording system. Global Oceans 2020: Singapore – U.S. Gulf Coast, 1–7. https://doi.org/10.1109/IEEECONF38699.2020.9388970

Leatherwood, S., and Walker, W. A. (1979). “The northern right whale dolphin Lissodelphis borealis Peale in the eastern North Pacific,” in Behavior of marine animals, Vol. 3: Cetaceans, edited by H. E. Winn and B. L. Olla (Plenum, New York), 85–141.

Rankin, S., Oswald, J., Barlow, J., & Lammers, M. (2007). Patterned burst-pulse vocalizations of the northern right whale dolphin, Lissodelphis borealis. The Journal of the Acoustical Society of America, 121(2), 1213–1218. https://doi.org/10.1121/1.2404919

Putting Fitbits on whales: How tag data allows for estimating calories burned by foraging PCFG gray whales

By: Kate Colson, MSc Student, University of British Columbia, Institute for the Oceans and Fisheries, Marine Mammal Research Unit

Hello! My name is Kate Colson and I am a master’s student at the University of British Columbia, co-supervised by Dr. Andrew Trites of the Marine Mammal Research Unit and Dr. Leigh Torres of the GEMM Lab. As part of my thesis work, I have had the opportunity to spend the summer field season with Leigh and the GEMM Lab team. 

For my master’s I am studying the foraging energetics of Pacific Coast Feeding Group (PCFG) gray whales as part of the much larger Gray whale Response to Ambient Noise Informed by Technology and Ecology (GRANITE) project. Quantifying the energy expenditure of PCFG gray whales during foraging can help establish a baseline for how disturbance impacts the ability of this unique population to meet their energy needs. Additionally, determining how many calories are burned during different PCFG foraging behaviors might help explain why some gray whales are in better body condition than others.

To understand how much energy different PCFG foraging behaviors cost, I am using data from suction cup tags we have temporarily applied on PCFG gray whales (Figure 1). You can read more about the why the GEMM Lab started using these tags in an earlier blog here. What I want to talk about in this blog is how exactly we can use this tag data to estimate energy expenditure of PCFG gray whales. 

Figure 1. The famous “Scarlett” with a suction cup tag just attached using a carbon fiber pole (seen on far right). This minimally invasive tag has many data sensors, all of which sample at high frequencies, that can allow for an estimation of energy expenditure for different gray whale behaviors. Source: GEMM Lab; National Marine Fisheries Service (NMFS) permit no. 21678 

The suction cups tags used in this project have many data sensors that are useful for describing the movement of the tagged whale including accelerometers, magnetometers, gyroscopes, and pressure sensors, and all are sampling at high frequencies. For example, the accelerometer is taking 400 measurements per second! The accelerometer, magnetometer, and gyroscope take measurements in 3 dimensions along the X, Y, and Z-axes. The whale’s movement around the X-axis indicates roll (if the whale is swimming on its side), while movement around the Y-axis indicates pitch (if the whales head is oriented towards the surface or the sea floor). Changes in the whale’s movement around the Z-axis indicates if the whale is changing its swimming direction. Together, all of these sensors can describe the dive profile, body orientation, fluking behavior, and fine-scale body movements of the animal down to the second (Figure 2). This allows for the behavior of the tagged whale to be specifically described for the entirety of the tag deployment. 

Figure 2. An example of what the tag sensor data looks like. The top panels show the depth of the animal and can be used to determine the diving behavior of the whale. The middle panels show the body roll of the whale (the X axis) —a roll value close to 0 means the whale is swimming “normally” with no rotation to either side, while a higher roll value means the whale is positioned on its side. The bottom panels show the fluking behavior of the animal: each spike is the whale using its tail to propel itself through the water, with higher spikes indicating a stronger fluke stroke. Source: GEMM Lab, NMFS permit no. 21678

Although these suction cup tags are a great advancement in collecting fine-scale data, they do not have a sensor that actually measures the whale’s metabolism, or rate of calories burned by the whale. Thus, to use this fine-scale tag data as an estimate for energy expenditure, a summary metric must be calculated from the data and used as a proxy. The most common metric found in the literature is Overall Dynamic Body Acceleration (ODBA) and many papers have been published discussing the pros and cons of using ODBA as a proxy for energy expenditure (Brown et al., 2013; Gleiss et al., 2011; Halsey, 2017; Halsey et al., 2011; Wilson et al., 2020). The theory behind ODBA is that because an animal’s metabolic rate is primarily comprised of movement costs, then measuring the acceleration of the body is an effective way of determining energy expenditure. This theory might seem very abstract, but if you have ever worn a Fitbit or similar fitness tracking device to estimate how many calories you’ve burned during a workout, the same principle applies. Those fitness devices use accelerometers and other sensors, to measure the movement of your limbs and produce estimates of energy used. 

So now that we’ve established that the goal of my research is to essentially use these suction cup tags as Fitbits for PCFG gray whales, let’s look at how accelerometry data has been used to detect foraging behavior in large whales so far. Many accelerometry tagging studies have used rorquals as a focal species (see Shadwick et al. (2019) for a review). Well-known rorqual species include humpback, fin, and blue whales. These species forage by using lunges to bulk feed on dense prey patches in the water column. Foraging lunges are indicated by isolated periods of high acceleration that are easily detectable in the tag data (Figure 3; Cade et al., 2016; Izadi et al., 2022). 

Figure 3. Top image: A foraging blue whale performing a surface lunge (Photo credit: GEMM Lab). Note the dense aggregation of krill in the whale’s mouth. Bottom image: The signature acceleration signal for lunge feeding (adapted from Izadi et al., 2022). Each color represents one of the 3D axes of whale movement. The discrete periods of high acceleration represent lunges. 

However, gray whales feed very differently from rorquals. Gray whales primarily suction feed on the benthos, using their head to dig into the sediment and filter prey out of the mud using their baleen. Yet,  PCFG gray whales often perform many other foraging behaviors such as headstanding and side-swimming (Torres et al., 2018). Additionally, PCFG gray whales tend to feed in water depths that are often shallower than their body length. This shallow depth makes it difficult to isolate signals of foraging in the accelerometry data from random variation in the data and separate the tag data into periods of foraging behaviors (Figure 4).

Figure 4. Top image: A foraging PCFG gray whale rolls on its side to feed on mysid prey. Bottom image: The graph shows the accelerometry data from our suction cup tags that can be used to calculate Overall Dynamic Body Acceleration (ODBA) as a way to estimate energy expenditure. Each color represents a different axis in the 3D motion of the whale. The X-axis is the horizontal axis shows forward and backward movement of the whale, the Y-axis shows the side-to-side movement of the whale, and the Z-axis shows the up-down motion of the whale. Note how there are no clear periods of high acceleration in all 3 axes simultaneously to indicate different foraging behaviors like is apparent during lunges of rorqual whales. However, there is a pattern showing that when acceleration in the Z-axis (blue line) is positive, the X- and Y-axes (red and green lines) are negative. Source: GEMM Lab; NMSF permit no. 21678

But there is still hope! Thanks to the GEMM Lab’s previous work describing the foraging behavior of the PCFG sub-group using drone footage, and the video footage available from the suction cup tags deployed on PCFG gray whales, the body orientation calculated from the tag data can be a useful indication of foraging. Specifically, high body roll is apparent in many foraging behaviors known to be used by the PCFG, and when the tag data indicates that the PCFG gray whale is rolled onto its sides, lots of sediment (and sometimes even swarms of mysid prey) is seen in the tag video footage. Therefore, I am busy isolating these high roll events in the collected tag data to identify specific foraging events. 

My next steps after isolating all the roll events will be to use other variables such as duration of the roll event and body pitch (i.e., if the whales head is angled down), to define different foraging behaviors present in the tag data. Then, I will use the accelerometry data to quantify the energetic cost of performing these behaviors, perhaps using ODBA. Hopefully when I visit the GEMM Lab again next summer, I will be ready to share which foraging behavior leads to PCFG gray whales burning the most calories!

References: 

Brown, D. D., Kays, R., Wikelski, M., Wilson, R., & Klimley, A. P. (2013). Observing the unwatchable through acceleration logging of animal behavior. Animal Biotelemetry, 1(1), 1–16. https://doi.org/10.1186/2050-3385-1-20

Cade, D. E., Friedlaender, A. S., Calambokidis, J., & Goldbogen, J. A. (2016). Kinematic diversity in rorqual whale feeding mechanisms. Current Biology, 26(19), 2617–2624. https://doi.org/10.1016/j.cub.2016.07.037

Duley, P. n.d. Fin whales feeding [photograph]. NOAA Northeast Fisheries Science Center Photo Gallery. https://apps-nefsc.fisheries.noaa.gov/rcb/photogallery/finback-whales.html

Gleiss, A. C., Wilson, R. P., & Shepard, E. L. C. (2011). Making overall dynamic body acceleration work: On the theory of acceleration as a proxy for energy expenditure. Methods in Ecology and Evolution, 2(1), 23–33. https://doi.org/10.1111/j.2041-210X.2010.00057.x

Halsey, L. G. (2017). Relationships grow with time: A note of caution about energy expenditure-proxy correlations, focussing on accelerometry as an example. Functional Ecology, 31(6), 1176–1183. https://doi.org/10.1111/1365-2435.12822

Halsey, L. G., Shepard, E. L. C., & Wilson, R. P. (2011). Assessing the development and application of the accelerometry technique for estimating energy expenditure. Comparative Biochemistry and Physiology – A Molecular and Integrative Physiology, 158(3), 305–314. https://doi.org/10.1016/j.cbpa.2010.09.002

Izadi, S., Aguilar de Soto, N., Constantine, R., & Johnson, M. (2022). Feeding tactics of resident Bryde’s whales in New Zealand. Marine Mammal Science, 1–14. https://doi.org/10.1111/mms.12918

Shadwick, R. E., Potvin, J., & Goldbogen, J. A. (2019). Lunge feeding in rorqual whales. Physiology, 34, 409–418. https://doi.org/10.1152/physiol.00010.2019

Torres, L. G., Nieukirk, S. L., Lemos, L., & Chandler, T. E. (2018). Drone up! Quantifying whale behavior from a new perspective improves observational capacity. Frontiers in Marine Science, 5, 1–14. https://doi.org/10.3389/fmars.2018.00319

Wilson, R. P., Börger, L., Holton, M. D., Scantlebury, D. M., Gómez-Laich, A., Quintana, F., Rosell, F., Graf, P. M., Williams, H., Gunner, R., Hopkins, L., Marks, N., Geraldi, N. R., Duarte, C. M., Scott, R., Strano, M. S., Robotka, H., Eizaguirre, C., Fahlman, A., & Shepard, E. L. C. (2020). Estimates for energy expenditure in free-living animals using acceleration proxies: A reappraisal. Journal of Animal Ecology, 89(1), 161–172. https://doi.org/10.1111/1365-2656.13040

Do you lose SLEAP over video analysis of gray whale behavior? Not us in the GEMM Lab! 

Celest Sorrentino, University of California, Santa Barbara, Department of Ecological, Evolution, and Marine Biology, GEMM Lab NSF REU intern

Are you thinking “Did anyone proofread this blog beforehand? Don’t they know how to spell SLEEP?”  I completely understand this concern, but not to fear: the spelling of SLEAP is intentional! We’ll address that clickbait in just a moment. 

My name is Celest Sorrentino, a first-generation Latina undergrad who leaped at the opportunity to depart from the beaches of Santa Barbara, California to misty Newport, Oregon to learn and grow as a scientist under the influential guidance of Clara Bird, Dr. Leigh Torres and the powerhouse otherwise known as the GEMM lab. As a recent NSF REU (Research Experience for Undergraduates) intern in the GEMM Lab at Oregon State University, I am thrilled to have the chance to finally let you in on the project Clara, Leigh and I have been working on all summer. Ready for this?

Our project uses a deep-learning platform called SLEAP A.I. ( https://sleap.ai/) that can predict and track multiple animals in video to track gray whale mother calf pairs in drone footage. We also took this project a step further and explored how the distance between a gray whale mother and her calf, a proxy for calf independence, varied throughout the season and by different calf characteristics. 

In this story, we’ve got a little bit for everyone: the dynamic duo of computer vision and machine learning for my data scientist friends, and ecological inquest for my cetacean researcher friends. 

About the Author

Before we begin, I’d like to share that I am not a data scientist. I’ve only ever taken one coding class. I also do not have years of gray whale expertise under my belt (not yet at least!). I’m entering my 5th year at University of California, Santa Barbara as a double major in Ecology and Evolution (BS) as well as Italian Studies (BA). I am sharing this information to convey the feasibility of learning how to use machine-learning as a solution to streamline the laborious task of video analysis, which would permit more time towards answering your own ecological question, as we did here.

Essential Background

Hundreds of Hours of Drone footage

Since 2016, the GEMM Lab has been collecting drone footage of gray whales off the Oregon Coast to observe gray whale behavior in more detail (Torres et al. 2018). Drones have been shown to increase observational time of gray whales by a three-fold (Torres et al. 2018), including the opportunity to revisit the video with fresh eyes at any time one pleases. The GEMM Lab has flow over 500 flights in the past 6 years, including limited footage of gray whale mother-calf pairs. Little is known about gray whale mother-calf dynamics and even less about factors that influence calf development. As we cannot interview hundreds of gray-whale mother-calf pairs to develop a baseline for this information, we explore potential proxies for calf development instead (similar to how developmental benchmarks are used for human growth). 

Distance and Development

During our own life journey, each of us became less and less dependent on our parents to survive on our own. Formulating our first words so that we can talk for ourselves, cracking an egg for our parents so that we can one day cook for ourselves, or even letting go of their hand when crossing the street. For humans, we spend many years with our kin preparing for these moments, but gray whale mother-calf pairs only have a few months after birth until they separate. Gray whale calves are born on their wintering grounds in Baja California, Mexico (around February), migrate north with their mothers to the foraging grounds, and are then weaned during the foraging season (we think around August). This short time with their mother means that they have to become independent pretty quickly (about 6 months!).

Distance between mother and calf can be considered a measure of independence because we would expect increased distance between the pair as calf independence increases. In a study by Nielson et al (2019), distance between Southern Right Whale mother-calf pairs was found to increase as the calf grew, indicating that it can serve as a good proxy for independence. The moment a mother-calf pair separates has not been documented, but the GEMM lab has footage of calves during the foraging season pre-weaning that can be used to investigate this process.  However, video analysis is no easy feat: video analysis can range from post-processing, diligent evaluation, and video documentation (Torres et al. 2018). Although the use of UAS has become a popular method for many researchers, the extensive time required for video analysis is a limitation. As mentioned in Clara’s blog, the choice to pursue different avenues to streamline this process, such as automation through machine learning, is highly dependent on the purpose and the kind of questions a project intends to answer.

SLEAP A.I.

 In a world where modern technology is constantly evolving to cater towards making everyday tasks easier, machine learning leads the charge with its ability for a machine to perform human tasks. Deep learning is a subset of machine learning in which the model learns and adapts the ability to perform a task given a dataset. SLEAP (Social LEAP Estimation of Animal Poses) A.I. is an open-source deep-learning framework created to be able to track multiple subjects, specifically animals, throughout a variety of environmental conditions and social dynamics. In previous cases, SLEAP has tracked animals with distinct morphologies and conditions such as mice interactions, fruit flies engaging in courtship, and bee behavior in a petri dish (Pereira 2020). While these studies show that SLEAP could help make video analysis more efficient, these experiments were all conducted on small animals and in controlled environments. However, large megafauna, such as gray whales, cannot be cultivated and observed in a controlled Petri dish. Could SLEAP learn and adapt to predict and track gray whales in an uncontrolled environment, where conditions are never the same (ocean visibility, sunlight, obstructions)? 

Methods

In order to establish a model within SLEAP, we split our mother-calf drone video dataset into training (n=9) and unseen/testing (n=3) videos. Training involves teaching the model to recognize gray whales, and necessitated me to label every four frames using the following labels (anatomical features): rostrum, blowhole, dorsal, dorsal-knuckle, and tail (Fig. 1). Once SLEAP was trained and able to successfully detect gray whales, we ran the model on unseen video. The purpose of using unseen video was to evaluate whether the model could adapt and perform on video it had never seen before, eliminating the need for a labeler to retrain it. 

We then extracted the pixel coordinates for the mom and calf, calculated the distance between their respective dorsal knuckles, and converted the distance to meters using photogrammetry (see KC’s blog  for a great explanation of these methods).  The distance between each pair was then summarized on a daily scale as the average distance and the standard deviation. Standard deviation was explored to understand how variable the distance between mother-calf pair was throughout the day. We then looked at how distance and the standard deviation of distance varied by day of year, calf Total Length (TL), and calf Body Area Index (BAI; a measure of body condition). We hypothesized that these three metrics may be drivers of calf independence (i.e., as the calf gets longer or fatter it becomes more independent from its mother).  

Fig 1. Example of a labelled frame from SLEAP, highlighting labels: rostrum, blowhole, dorsal, dorsal-knuckle, and tail. 

Results

SLEAP A.I. was able to successfully detect and track gray whale mother-calf pairs across all videos (that’s a total of 1318 frames!). When evaluating how the average distance changed across Day of Year, calf Total length, and calf BAI, the plots did not demonstrate the positive relationship we anticipated (Fig 2A). However, when evaluating the standard deviation of distance across Day of Year, calf Total Length, and calf BAI, we did notice that there does appear to be an increase in variability of distance with an increase in Day of Year and calf Total length (Fig 2B)

Fig 2A: Relationship between average distance(m) between each mother and calf (colors represent different named calves) relative to Day of Year, Total length of calf, and BAI of calf. 
Fig 2B: Relationship between standard deviation of  distance(m) between each mother and calf (colors represent different named calves) relative to Day of Year, Total length of calf, and BAI of calf.

Concluding thoughts

These results are monumental! We demonstrated the feasibility to use AI to create a model that can track gray whale pairs in drone footage, which is a fantastic tool that can be applied to updated datasets in the future. As more footage of gray whale mother-calf pairs are collected, this video can be quickly uploaded to SLEAP for model evaluation, predictions can be exported, and results subsequently included in the distance analysis to update our plots and increase our understanding. Our data currently provide a preliminary understanding of how the distance between mother-calf pairs changes with Day of Year, Total length, and BAI, but we are now able to continue updating our dataset as we collect more drone footage. 

I suppose you can say I did mislead you a bit with my title, as I have lost some SLEEP recently. But, not over video analysis per say but rather in the form of inspiration. Inspiration toward expanding my understanding of machine learning so that it can be applied toward answering pressing ecological questions. This project has only propelled me to dig my heels in and investigate further the potential of machine learning to analyze dense datasets for huge knowledge gains.

Fig 3A: Snapshot of Celest working in SLEAP GUI.

Acknowledgements

This project was made possible in partnership by the continuous support by Clara Bird, Dr. Leigh Torres, KC Bierlich, and the entire GEMM Lab!

References

Nielsen, M., Sprogis, K., Bejder, L., Madsen, P., & Christiansen, F. (2019). Behavioural development in southern right whale calves. Marine Ecology Progress Series, 629, 219–234. https://doi.org/10.3354/meps13125

Pereira, Talmo D., Nathaniel Tabris, Junyu Li, Shruthi Ravindranath, Eleni S. Papadoyannis, Z. Yan Wang, David M. Turner, et al. “SLEAP: Multi-Animal Pose Tracking.” Preprint. Animal Behavior and Cognition, September 2, 2020. https://doi.org/10.1101/2020.08.31.276246.

Torres, Nieukirk, S. L., Lemos, L., & Chandler, T. E. (2018). Drone Up! Quantifying Whale Behavior From a New Perspective Improves Observational Capacity. Frontiers in Marine Science, 5. https://doi.org/10.3389/fmars.2018.00319