Tag: social behavior

That’s so Real: Adult Beginners, Serial Podcast(s), and a whole lotta of Baja Gray Whale Video Analysis.

Celest Sorrentino, Research Technician, Geospatial Ecology of Marine Megafauna Lab

Hello again GEMM Lab family. I write to you exactly a year after (okay maybe 361 days after but who’s counting…) from my previous blog post describing my 2022 summer working in the GEMM Lab as an NSF REU intern. Since then, so much has changed, and I can’t wait to fill you in on it.

In June I walked across the commencement stage at UC Santa Barbara, earning my BS in Ecology, Evolution, and Marine Biology and my minor in Italian language. A week later, I packed my bags and headed straight back to the lukewarm beaches of Newport, Oregon as a Research Technician in the GEMM Lab. I am incredibly fortunate to have been invited back to the OSU Marine Mammal Institute to lend a hand analyzing drone footage of gray whales collected back in March 2023 when Leigh and Clara went down to Baja California, as mentioned previously in Clara’s blog. 

Fig. 1. View from the top! (of the bridge at Yaquina Bay Bridge in Newport, OR)

During my first meeting with Clara at the beginning of the summer we discussed that a primary goal of my position was to process all the drone footage collected in Baja so that the generated video clips could be later used in other analytical software such as BORIS and SLEAP A.I. Given my previous internships and past summer project, this video processing is familiar to me. My initial thoughts were:

Sweet! Watch drone footage, pop in some podcasts, note down when I see whales, let’s do this!*

Like any overly eager 23-year-old, I might have mentally cracked open a Celsius and kicked my feet up too soon. We added another layer to the goal: develop an ethogram – which requires me to identify and define the behaviors that the gray whales appear to be demonstrating within the videos (more on ethogram development in Clara’s previous blog.) This made me nervous. 

I don’t have any experience with behavior. How do I tell what is a real behavior or if the whale is just existing? What if I’m wrong and ruin the project? What if I totally mess this up?

Naturally, as any sane person, to resolve these thoughts I took to the Reddit search bar: “How to do a job you’ve never done before.” No dice. 

I pushed these thoughts aside and decided to just start the video analysis process. Clara provided me with the ethogram she is developing during her PhD as a point of reference (based on the published gray whale ethogram in Torres et al. 2018), I was surrounded by an insanely supportive lab, and I could Google anything at my fingertips. Fast-forward 6 weeks later: I had analyzed 128 drone videos of adult gray whales as well as mother-calf pairs, and developed an ethogram describing, 26 behaviors**. I named one of my favorite behaviors  a “Twirl” to describe when a gray whale lifts their head out of the water and performs a 360 turn. Reminds me of times when as a kid, sometimes all you really needed is a good spin!

Now I was ready to start a productive, open conversation with Leigh and Clara about this ethogram and my work. However, even walking up to that last meeting, remnants of those daunting, doubtful early summer thoughts persisted. Even after I double checked all the definitions I wrote, rewatched all videos with said behaviors, and had something to show for my work. What gives Brain?

A few days ago, as I sat on my family’s living room couch with my two younger sisters, Baylie and Cassey, Baylie wanted to watch some TikToks with me. One video that came up was of a group of adults taking a beginner dance class, having so much fun and radiating joy. The caption read, Being a beginner as an adult is such a fun and wild thing. Baylie and I watched the video at least 10x, repeating to each other phrases like, “Wow!” and “They’re so cool.” That caption and video has been on my mind since: 

Being a beginner as an adult is such a fun and wild thing.

Being a beginner as an adult is also scary. 

Having just graduated, I can no longer say I am undergraduate student. Now, I am a young adult. This was my first research technician job, as an adult. Don’t adults usually have everything figured out? Can adults be beginners too?

Yes. In fact, we’re beginners more than we realize. 

  • I was a beginner cooking my mother’s turkey recipe 3 years ago for my housemates during the pandemic (Even after having her on Facetime, I still managed to broil it a little too long.) 
  • I was a beginner driver 5 years ago in a rickety Jeep driving myself to school (Now, since I’ve been back home, I’ve been driving my little sisters to school.)
  • I was a beginner NSF REU intern just a year ago. (This summer I was the alumni on the panel for the current NSF REU interns at Hatfield.)
  • I was a beginner science communicator presenting my NSF REU project at Hatfield last summer. (This summer, I presented my research at the Animal Behavior Society Conference.) 
Fig 2A. Group Pic with the LABIRINTO Lab and GEMM Lab at the ABS Portland Conference!
Fig 2B. Clara Bird (left), Dr. Leigh Torres (middle), and I (right) at the ABS Portland Conference. 

I now recognize that during my time identifying and defining behaviors of gray whales in videos made me take on the seat of a “beginner video and behavioral analyst”. I could not rely on the automated computer vision lens I gained from previous internships, which felt familiar and secure. 

 Instead, I had to allow myself to be creative. Dig into the unfamiliar in an effort to complete a task or job I had never done before. Allowing myself to be imperfect, make mistakes, meanwhile unconsciously building a new skill. 

This is what makes being a beginner as an adult such a fun thing. 

I don’t think being a beginner is a wild thing, although it can definitely make you feel a wild range of emotions. Being a beginner means you’re allowing yourself to try something new. Being a beginner means you’re allowing yourself the chance to learn.

Whether you’re an adult beginner as you enter your 30s, adult beginner as you enter parenthood, adult beginner grabbing a drink with friends after a long day in lab, adult beginner as a dancer, or like me, a beginner of leaving behind my college student persona and entering a new identity of adulthood, being a beginner as an adult is such a fun and normal thing.

I am not sure what will be next, but I hope to write to you all again from this blog a year from now, as an adult beginner as a grad student in the GEMM Lab. For anyone approaching the question of “What’s next”, I encourage you to read “Never a straight Path” by GEMM Lab MSc alum Florence Sullivan, a blog that has brought me such solace in my new adult journey and advice that never gets old.

Being a beginner—that, is so real. 

Fig 3A. Kayaking as an adult beginner of the Port Orford Field Team!
Fig 3B “See you soon:” Wolftree evenings with the lab.
Fig 3C. GEMM Lab first BeReal!

*I listened to way too many podcasts to list them all, but I will include two that have been a GEMM Lab “gem” —-thanks to Lisa and Clara for looping me in and now, looping you in!)

**(subject to change)

References

Torres LG, Nieukirk SL, Lemos L, Chandler TE (2018) Drone Up! Quantifying Whale Behavior From a New Perspective Improves Observational Capacity. Front Mar Sci 510.3389/fmars.2018.00319

Did you enjoy this blog? Want to learn more about marine life, research, and conservation? Subscribe to our blog and get a weekly message when we post a new blog. Just add your name and email into the subscribe box below.

Do you lose SLEAP over video analysis of gray whale behavior? Not us in the GEMM Lab! 

Celest Sorrentino, University of California, Santa Barbara, Department of Ecological, Evolution, and Marine Biology, GEMM Lab NSF REU intern

Are you thinking “Did anyone proofread this blog beforehand? Don’t they know how to spell SLEEP?”  I completely understand this concern, but not to fear: the spelling of SLEAP is intentional! We’ll address that clickbait in just a moment. 

My name is Celest Sorrentino, a first-generation Latina undergrad who leaped at the opportunity to depart from the beaches of Santa Barbara, California to misty Newport, Oregon to learn and grow as a scientist under the influential guidance of Clara Bird, Dr. Leigh Torres and the powerhouse otherwise known as the GEMM lab. As a recent NSF REU (Research Experience for Undergraduates) intern in the GEMM Lab at Oregon State University, I am thrilled to have the chance to finally let you in on the project Clara, Leigh and I have been working on all summer. Ready for this?

Our project uses a deep-learning platform called SLEAP A.I. ( https://sleap.ai/) that can predict and track multiple animals in video to track gray whale mother calf pairs in drone footage. We also took this project a step further and explored how the distance between a gray whale mother and her calf, a proxy for calf independence, varied throughout the season and by different calf characteristics. 

In this story, we’ve got a little bit for everyone: the dynamic duo of computer vision and machine learning for my data scientist friends, and ecological inquest for my cetacean researcher friends. 

About the Author

Before we begin, I’d like to share that I am not a data scientist. I’ve only ever taken one coding class. I also do not have years of gray whale expertise under my belt (not yet at least!). I’m entering my 5th year at University of California, Santa Barbara as a double major in Ecology and Evolution (BS) as well as Italian Studies (BA). I am sharing this information to convey the feasibility of learning how to use machine-learning as a solution to streamline the laborious task of video analysis, which would permit more time towards answering your own ecological question, as we did here.

Essential Background

Hundreds of Hours of Drone footage

Since 2016, the GEMM Lab has been collecting drone footage of gray whales off the Oregon Coast to observe gray whale behavior in more detail (Torres et al. 2018). Drones have been shown to increase observational time of gray whales by a three-fold (Torres et al. 2018), including the opportunity to revisit the video with fresh eyes at any time one pleases. The GEMM Lab has flow over 500 flights in the past 6 years, including limited footage of gray whale mother-calf pairs. Little is known about gray whale mother-calf dynamics and even less about factors that influence calf development. As we cannot interview hundreds of gray-whale mother-calf pairs to develop a baseline for this information, we explore potential proxies for calf development instead (similar to how developmental benchmarks are used for human growth). 

Distance and Development

During our own life journey, each of us became less and less dependent on our parents to survive on our own. Formulating our first words so that we can talk for ourselves, cracking an egg for our parents so that we can one day cook for ourselves, or even letting go of their hand when crossing the street. For humans, we spend many years with our kin preparing for these moments, but gray whale mother-calf pairs only have a few months after birth until they separate. Gray whale calves are born on their wintering grounds in Baja California, Mexico (around February), migrate north with their mothers to the foraging grounds, and are then weaned during the foraging season (we think around August). This short time with their mother means that they have to become independent pretty quickly (about 6 months!).

Distance between mother and calf can be considered a measure of independence because we would expect increased distance between the pair as calf independence increases. In a study by Nielson et al (2019), distance between Southern Right Whale mother-calf pairs was found to increase as the calf grew, indicating that it can serve as a good proxy for independence. The moment a mother-calf pair separates has not been documented, but the GEMM lab has footage of calves during the foraging season pre-weaning that can be used to investigate this process.  However, video analysis is no easy feat: video analysis can range from post-processing, diligent evaluation, and video documentation (Torres et al. 2018). Although the use of UAS has become a popular method for many researchers, the extensive time required for video analysis is a limitation. As mentioned in Clara’s blog, the choice to pursue different avenues to streamline this process, such as automation through machine learning, is highly dependent on the purpose and the kind of questions a project intends to answer.

SLEAP A.I.

 In a world where modern technology is constantly evolving to cater towards making everyday tasks easier, machine learning leads the charge with its ability for a machine to perform human tasks. Deep learning is a subset of machine learning in which the model learns and adapts the ability to perform a task given a dataset. SLEAP (Social LEAP Estimation of Animal Poses) A.I. is an open-source deep-learning framework created to be able to track multiple subjects, specifically animals, throughout a variety of environmental conditions and social dynamics. In previous cases, SLEAP has tracked animals with distinct morphologies and conditions such as mice interactions, fruit flies engaging in courtship, and bee behavior in a petri dish (Pereira 2020). While these studies show that SLEAP could help make video analysis more efficient, these experiments were all conducted on small animals and in controlled environments. However, large megafauna, such as gray whales, cannot be cultivated and observed in a controlled Petri dish. Could SLEAP learn and adapt to predict and track gray whales in an uncontrolled environment, where conditions are never the same (ocean visibility, sunlight, obstructions)? 

Methods

In order to establish a model within SLEAP, we split our mother-calf drone video dataset into training (n=9) and unseen/testing (n=3) videos. Training involves teaching the model to recognize gray whales, and necessitated me to label every four frames using the following labels (anatomical features): rostrum, blowhole, dorsal, dorsal-knuckle, and tail (Fig. 1). Once SLEAP was trained and able to successfully detect gray whales, we ran the model on unseen video. The purpose of using unseen video was to evaluate whether the model could adapt and perform on video it had never seen before, eliminating the need for a labeler to retrain it. 

We then extracted the pixel coordinates for the mom and calf, calculated the distance between their respective dorsal knuckles, and converted the distance to meters using photogrammetry (see KC’s blog  for a great explanation of these methods).  The distance between each pair was then summarized on a daily scale as the average distance and the standard deviation. Standard deviation was explored to understand how variable the distance between mother-calf pair was throughout the day. We then looked at how distance and the standard deviation of distance varied by day of year, calf Total Length (TL), and calf Body Area Index (BAI; a measure of body condition). We hypothesized that these three metrics may be drivers of calf independence (i.e., as the calf gets longer or fatter it becomes more independent from its mother).  

Fig 1. Example of a labelled frame from SLEAP, highlighting labels: rostrum, blowhole, dorsal, dorsal-knuckle, and tail. 

Results

SLEAP A.I. was able to successfully detect and track gray whale mother-calf pairs across all videos (that’s a total of 1318 frames!). When evaluating how the average distance changed across Day of Year, calf Total length, and calf BAI, the plots did not demonstrate the positive relationship we anticipated (Fig 2A). However, when evaluating the standard deviation of distance across Day of Year, calf Total Length, and calf BAI, we did notice that there does appear to be an increase in variability of distance with an increase in Day of Year and calf Total length (Fig 2B)

Fig 2A: Relationship between average distance(m) between each mother and calf (colors represent different named calves) relative to Day of Year, Total length of calf, and BAI of calf. 
Fig 2B: Relationship between standard deviation of  distance(m) between each mother and calf (colors represent different named calves) relative to Day of Year, Total length of calf, and BAI of calf.

Concluding thoughts

These results are monumental! We demonstrated the feasibility to use AI to create a model that can track gray whale pairs in drone footage, which is a fantastic tool that can be applied to updated datasets in the future. As more footage of gray whale mother-calf pairs are collected, this video can be quickly uploaded to SLEAP for model evaluation, predictions can be exported, and results subsequently included in the distance analysis to update our plots and increase our understanding. Our data currently provide a preliminary understanding of how the distance between mother-calf pairs changes with Day of Year, Total length, and BAI, but we are now able to continue updating our dataset as we collect more drone footage. 

I suppose you can say I did mislead you a bit with my title, as I have lost some SLEEP recently. But, not over video analysis per say but rather in the form of inspiration. Inspiration toward expanding my understanding of machine learning so that it can be applied toward answering pressing ecological questions. This project has only propelled me to dig my heels in and investigate further the potential of machine learning to analyze dense datasets for huge knowledge gains.

Fig 3A: Snapshot of Celest working in SLEAP GUI.

Acknowledgements

This project was made possible in partnership by the continuous support by Clara Bird, Dr. Leigh Torres, KC Bierlich, and the entire GEMM Lab!

References

Nielsen, M., Sprogis, K., Bejder, L., Madsen, P., & Christiansen, F. (2019). Behavioural development in southern right whale calves. Marine Ecology Progress Series, 629, 219–234. https://doi.org/10.3354/meps13125

Pereira, Talmo D., Nathaniel Tabris, Junyu Li, Shruthi Ravindranath, Eleni S. Papadoyannis, Z. Yan Wang, David M. Turner, et al. “SLEAP: Multi-Animal Pose Tracking.” Preprint. Animal Behavior and Cognition, September 2, 2020. https://doi.org/10.1101/2020.08.31.276246.

Torres, Nieukirk, S. L., Lemos, L., & Chandler, T. E. (2018). Drone Up! Quantifying Whale Behavior From a New Perspective Improves Observational Capacity. Frontiers in Marine Science, 5. https://doi.org/10.3389/fmars.2018.00319

What drives individual specialization?

Clara Bird, PhD Student, OSU Department of Fisheries, Wildlife, and Conservation Sciences, Geospatial Ecology of Marine Megafauna Lab

When I wrote my first blog on individual specialization well over a year ago, I just skimmed the surface of the literature on this topic and only started to recognize the importance of studying individual specialization. The question, “is there individual specialization in the PCFG of gray whales?” is the focus of my first thesis chapter and the results will affect all my subsequent work. Therefore, the literature and concepts of individual specialization are a focus of my literature review and studies.

In my previous blog I focused on common characteristics of individuals that are similarly specialized as an underlying driver of individual specialization. While these characteristics (often attributable to age, sex, or physical traits) are important to consider, I’ve learned that the list of drivers of individual specialization is long and that many variables are dynamic. Of all the drivers I’ve learned about, competition is among the most common.

Competition is a major driver of individual specialization, and a common driver of competition is resource availability. When resource availability decreases, whether caused by increasing population density or changing environmental conditions, competition for that resource increases. As competition increases, individuals have a choice. They can choose to engage in competition, either by racing, fighting, or sharing [1], which can be costly, or they can diffuse the competition by focusing on a different resource.  This second approach would be considered niche partitioning in the prey dimension. Niche partitioning is a way for individuals to share ecological space by using different resources. Essentially, individuals can share habitat without having to engage in direct competition by pursuing different prey types [2]. 

This switch to different prey types can change the degree of individual specialization present in the population (Figure 1). But the direction of the change is not constant. If all individuals were pursuing the same prey type under low competition conditions but then switched to different alternate prey types under high competition, then individual specialization would increase (Figure 1a). This direction has been observed across a range of species including sharks [3], otters [4]–[7], dolphins [8], [9], stickleback fish [10], [11], largemouth bass [12], banded mongoose [13], fur seals [14], and baleen whales [15].

However, if individuals were pursuing different prey types under low competition conditions (maybe because of underlying differences such as age or sex) but then switched to the same alternate prey types under high competition, diet overlap would increase, and individual specialization would decrease (Figure 1b). Furthermore, an individual might not switch to an entirely new prey type but instead add prey items to its diet [16]. This diet expansion under competition would also decrease individual specialization. Fewer studies have reported this direction but it’s been found in the common bumblebee [17] and in several neotropical vertebrate species [18], [19].

Figure `1. Figure 3 from Araújo et al. 2011 [20]. Illustration of how ecological mechanisms may affect the degree of individual specialization. Arrows linking resources to individual consumers indicate resource consumption (relative thickness indicates proportional contribution). 
Horizontal arrows indicate the sign (positive or negative) of the effect on the degree of individual specialization. (a) Consumers share the same preferred resource (dark gray tangle) but have different alternative resources (white and light gray triangles). As the preferred resource becomes scarce, consumers switch to different alternatives, increasing the degree of individual specialization. (b) Alternatively, consumers have distinct preferred resources, so that as resources become scarce, individuals converge to the alternative resource (dark gray triangle), reducing diet variation.

Interestingly, its hypothesized that individual specialization driven by competition is one of the factors that facilitates the formation and existence of stable groups [21]. For example, a study on resident female dolphins in Sarasota Bay, FL, USA found that females with high spatial overlap used distinct foraging specializations [8](Figure 2). This study illustrates how partitioning prey enabled spatial and social coexistence. A study on banded mongooses reached a similar conclusion [13]. They found that specialization was highest in the biggest groups (with the most competition) and not explained by sex, age, or other inherent differences. They hypothesized that specialization increasing with competition reduced conflict and allowed the groups to remain stable. This study also highlighted the role of learning to determine an individual’s specialization.

Figure 2. A bottlenose dolphin.
Source: https://sarasotadolphin.org

Learning drives the distribution of knowledge throughout a population, which can lead to either specialization or generalization. ‘One-to-one’ learning, where one individual learns from one demonstrator, tends to promote individual specialization [21]. This form of transmission drives specialization because the individuals who learn the specialization tend to then carry on using, and eventually teaching, that specialization [6]. A common example of ‘one-to-one’ learning is vertical transmission from parent to offspring. It has been shown to transmit specializations in dolphins [22] and otters [6]. ‘One-to-one’ learning can occur outside of parent-offspring pairs; non-parent-offspring ‘one-to-one’ learning has been shown to drive specialization in banded mongooses [13](Figure 3).

However, other forms of social learning can promote more generalized foraging strategies. ‘Many-to-one’ or ‘one-to-many’ learning  can reduce the presence of specialization in species [13], [21] as can the presence of conformity in a group [23], [24].

Figure 3. A group of banded mongooses.
Source: http://socialisresearch.org/about-the-banded-mongoose-project/

The multiple drivers of specialization and their dynamic quality means that it is important to contextualize specialization. For example, a study on four species of neotropical frogs found varying degrees of specialization across multiple populations of each species [18]. The degree of specialization was dependent on a variety of drivers including predation and both intra- and inter-specific competition. Notably, the direction of the relationship between degree of specialization and each driver was species specific. This study highlights that one species may not always be more specialized than another, but that a populations’ specialization is context dependent.

Therefore, it is important to not only be aware of the degree of specialization present in a population, but to also understand its dynamics and drivers. These relationships can then be used to understand how, and why, a population may react to competition from other species, predators, and changes in resource availability [20].  A population’s specialization can also affect the specialization of other populations and community dynamics [25], therefore, it’s important to consider and study individual specialization on both the population and community level. I am excited to start using our incredible six-year dataset to start investigating these questions for PCFG gray whales, so stay tuned for results!

Did you enjoy this blog? Want to learn more about marine life, research, and conservation? Subscribe to our blog and get a weekly alert when we make a new post! Just add your name into the subscribe box on the left panel.  

References

[1]       M. Taborsky, M. A. Cant, and J. Komdeur, The Evolution of Social Behaviour. Cambridge: Cambridge University Press, 2021. doi: 10.1017/9780511894794.

[2]       E. R. Pianka, “Niche Overlap and Diffuse Competition,” vol. 71, no. 5, pp. 2141–2145, 1974.

[3]       P. Matich et al., “Ecological niche partitioning within a large predator guild in a nutrient-limited estuary,” Limnol. Oceanogr., vol. 62, no. 3, pp. 934–953, 2017, doi: https://doi.org/10.1002/lno.10477.

[4]       S. D. Newsome et al., “The interaction of intraspecific competition and habitat on individual diet specialization: a near range-wide examination of sea otters,” Oecologia, vol. 178, no. 1, pp. 45–59, May 2015, doi: 10.1007/s00442-015-3223-8.

[5]       M. T. Tinker, G. Bentall, and J. A. Estes, “Food limitation leads to behavioral diversification and dietary specialization in sea otters,” Proc. Natl. Acad. Sci., vol. 105, no. 2, pp. 560–565, Jan. 2008, doi: 10.1073/pnas.0709263105.

[6]       M. T. Tinker, M. Mangel, and J. A. Estes, “Learning to be different: acquired skills, social learning, frequency dependence, and environmental variation can cause behaviourally mediated foraging specializations,” Evol. Ecol. Res., vol. 11, pp. 841–869, 2009.

[7]       M. T. Tinker et al., “Structure and mechanism of diet specialisation: testing models of individual variation in resource use with sea otters,” Ecol. Lett., vol. 15, no. 5, pp. 475–483, 2012, doi: 10.1111/j.1461-0248.2012.01760.x.

[8]       S. Rossman et al., “Foraging habits in a generalist predator: Sex and age influence habitat selection and resource use among bottlenose dolphins (Tursiops truncatus),” Mar. Mammal Sci., vol. 31, no. 1, pp. 155–168, 2015, doi: https://doi.org/10.1111/mms.12143.

[9]       L. G. Torres, “A kaleidoscope of mammal , bird and fish : habitat use patterns of top predators and their prey in Florida Bay,” vol. 375, pp. 289–304, 2009, doi: 10.3354/meps07743.

[10]     M. S. Araújo et al., “Network Analysis Reveals Contrasting Effects of Intraspecific Competition on Individual Vs. Population Diets,” Ecology, vol. 89, no. 7, pp. 1981–1993, 2008, doi: 10.1890/07-0630.1.

[11]     R. Svanbäck and D. I. Bolnick, “Intraspecific competition drives increased resource use diversity within a natural population,” Proc. R. Soc. B Biol. Sci., vol. 274, no. 1611, pp. 839–844, Mar. 2007, doi: 10.1098/rspb.2006.0198.

[12]     D. E. Schindler, J. R. Hodgson, and J. F. Kitchell, “Density-dependent changes in individual foraging specialization of largemouth bass,” Oecologia, vol. 110, no. 4, pp. 592–600, May 1997, doi: 10.1007/s004420050200.

[13]     C. E. Sheppard et al., “Intragroup competition predicts individual foraging specialisation in a group-living mammal,” Ecol. Lett., vol. 21, no. 5, pp. 665–673, 2018, doi: 10.1111/ele.12933.

[14]     L. Kernaléguen, J. P. Y. Arnould, C. Guinet, and Y. Cherel, “Determinants of individual foraging specialization in large marine vertebrates, the Antarctic and subantarctic fur seals,” J. Anim. Ecol., vol. 84, no. 4, pp. 1081–1091, 2015, doi: 10.1111/1365-2656.12347.

[15]     E. M. Keen and K. M. Qualls, “Respiratory behaviors in sympatric rorqual whales: the influence of prey depth and implications for temporal access to prey,” J. Mammal., vol. 99, no. 1, pp. 27–40, Feb. 2018, doi: 10.1093/jmammal/gyx170.

[16]     R. H. MacArthur and E. R. Pianka, “On Optimal Use of a Patchy Environment,” Am. Nat., vol. 100, no. 916, pp. 603–609, 1966, doi: 10.1086/282454.

[17]     C. Fontaine, C. L. Collin, and I. Dajoz, “Generalist foraging of pollinators: diet expansion at high density,” J. Ecol., vol. 96, no. 5, pp. 1002–1010, 2008, doi: 10.1111/j.1365-2745.2008.01405.x.

[18]     R. Costa-Pereira, V. H. W. Rudolf, F. L. Souza, and M. S. Araújo, “Drivers of individual niche variation in coexisting species,” J. Anim. Ecol., vol. 87, no. 5, pp. 1452–1464, 2018, doi: 10.1111/1365-2656.12879.

[19]     M. M. Pires, P. R. Guimarães Jr, M. S. Araújo, A. A. Giaretta, J. C. L. Costa, and S. F. dos Reis, “The nested assembly of individual-resource networks,” J. Anim. Ecol., vol. 80, no. 4, pp. 896–903, 2011, doi: 10.1111/j.1365-2656.2011.01818.x.

[20]     M. S. Araújo, D. I. Bolnick, and C. A. Layman, “The ecological causes of individual specialisation,”Ecol. Lett., vol. 14, no. 9, pp. 948–958, 2011, doi: https://doi.org/10.1111/j.1461-0248.2011.01662.x.

[21]     C. E. Sheppard, R. Heaphy, M. A. Cant, and H. H. Marshall, “Individual foraging specialization in group-living species,” Anim. Behav., vol. 182, pp. 285–294, Dec. 2021, doi: 10.1016/j.anbehav.2021.10.011.

[22]     S. Wild, S. J. Allen, M. Krützen, S. L. King, L. Gerber, and W. J. E. Hoppitt, “Multi-network-based diffusion analysis reveals vertical cultural transmission of sponge tool use within dolphin matrilines,” Biol. Lett., vol. 15, no. 7, p. 20190227, Jul. 2019, doi: 10.1098/rsbl.2019.0227.

[23]     L. M. Aplin, D. R. Farine, J. Morand-Ferron, A. Cockburn, A. Thornton, and B. C. Sheldon, “Experimentally induced innovations lead to persistent culture via conformity in wild birds,” Nature, vol. 518, no. 7540, pp. 538–541, Feb. 2015, doi: 10.1038/nature13998.

[24]     E. Van de Waal, C. Borgeaud, and A. Whiten, “Potent Social Learning and Conformity Shape a Wild Primate’s Foraging Decisions,” Science, Apr. 2013, doi: 10.1126/science.1232769.

[25]     D. I. Bolnick et al., “Why intraspecific trait variation matters in community ecology,” Trends Ecol. Evol., vol. 26, no. 4, pp. 183–192, Apr. 2011, doi: 10.1016/j.tree.2011.01.009.

The costs and benefits of automated behavior classification

Clara Bird, PhD Student, OSU Department of Fisheries, Wildlife, and Conservation Sciences, Geospatial Ecology of Marine Megafauna Lab

“Why don’t you just automate it?” This is a question I am frequently asked when I tell someone about my work. My thesis involves watching many hours of drone footage of gray whales and meticulously coding behaviors, and there are plenty of days when I have asked myself that very same question. Streamlining my process is certainly appealing and given how wide-spread and effective machine learning methods have become, it is a tempting option to pursue. That said, machine learning is only appropriate for certain research questions and scales, and it’s important to consider these before investing in using a new tool.

The application of machine learning methods to behavioral ecology is called computational ethology (Anderson & Perona, 2014). To identify behaviors from videos, the model tracks individuals across video frames and identifies patterns of movement that form a behavior. This concept is similar to the way we identify a whale as traveling if it’s moving in a straight line and as foraging if it’s swimming in circles within a small area (Mayo & Marx, 1990, check out this blog to learn more). The level of behavioral detail that the model is able to track  depends on the chosen method (Figure 1, Pereira et al., 2020). These methods range from tracking each animal as a simple single point (called a centroid) to tracking the animal’s body positioning in 3D (this method is called pose estimation), which range from providing less detailed to more detailed behavior definitions. For example, tracking an individual as a centroid could be used to classify traveling and foraging behaviors, while pose estimation could identify specific foraging tactics. 

Figure 1. Figure from Pereira et al. (2020) illustrating the different methods of animal behavior tracking that are possible using machine learning.

Pose estimation involves training the machine learning algorithm to track individual anatomical features of an individual (e.g., the head, legs, and tail of a rat), meaning that it can define behaviors in great detail. A behavior state could be defined as a combination of the angle between the tail and the head, and the stride length. 

For example, Mearns et al. (2020) used pose estimation to study how zebrafish larvae in a lab captured their prey. They tracked the tail movements of individual larvae when presented with prey and classified these movements into separate behaviors that allowed them to associate specific behaviors with prey capture (Figure 2). The authors found that these behaviors occurred in a specific sequence, that the behaviors kept the prey within the larvae’s line of sight, and that the sequence was triggered by visual cues.  In fact, when they removed the visual cue of the prey, the larvae terminated the behavior sequence, meaning that the larvae are continually choosing to do each behavior in the sequence, rather than the sequence being one long behavior event that is triggered only by the initial visual cue. This study is a good example of the applicability of machine learning models for questions aimed at kinematics and fine-scale movements. Pose estimation has also been used to study the role of facial expression and body language in rat social communication (Ebbesen & Froemke, 2021). 

Figure 2. Excerpt from figure 1 of Mearns et al. (2020) illustrating (A) the camera set up for their experiment, (B) how the model tracked the eye angles and tail of the larvae fish, (C) the kinematics extracted from the footage. In panel (C) the top plot shows how the eyes converged on the same object (the prey) during prey capture event, the middle plot shows when the tail was curved to the left or the right, and the bottom plot shows the angle of the tail tip relative to the body.

While previous machine learning methods to track animal movements required individuals to be physically marked, the current methods can perform markerless tracking (Pereira et al., 2020). This improvement has broadened the kinds of studies that are possible. For example, Bozek et al., (2021) developed a model that tracked individuals throughout an entire honeybee colony and showed that certain individual behaviors were spatially distributed within the colony (Figure 3). Machine learning enabled the researchers to track over 1000 individual bees over several months, a task that would be infeasible for someone to do by hand. 

Figure 3. Excerpt from figure 1 of Bozek et al., (2021) showing how individual bees and their trajectories were tracked.

These studies highlight that the potential benefits of using machine learning when studying fine scale behaviors (like kinematics) or when tracking large groups of individuals. Furthermore, once it’s trained, the model can process large quantities of data in a standardized way to free up time for the scientists to focus on other tasks.

While machine learning is an exciting and enticing tool, automating behavior detection via machine learning could be its own PhD dissertation. Like most things in life, there are costs and benefits to using this technique. It is a technically difficult tool, and while applications exist to make it more accessible, knowledge of the computer science behind it is necessary to apply it effectively and correctly. Secondly, it can be tedious and time consuming to create a training dataset for the model to “learn” what each behavior looks like, as this step involves manually labeling examples for the model to use. 

As I’ve mentioned in a previous blog, I came quite close to trying to study the kinematics of gray whale foraging behaviors but ultimately decided that counting fluke beats wasn’t necessary to answer my behavioral research questions. It was important to consider the scale of my questions (as described in Allison’s blog) and I think that diving into more fine-scale kinematics questions could be a fascinating follow-up to the questions I’m asking in my PhD. 

For instance, it would be interesting to quantify how gray whales use their flukes for different behavior tactics. Do gray whales in better body condition beat their flukes more frequently while headstanding? Does the size of the fluke affect how efficiently they can perform certain tactics? While these analyses would help quantify the energetic costs of different behaviors in better detail, they aren’t necessary for my broad scale questions. Consequently, taking the time to develop and train a pose estimation machine learning model is not the best use of my time.

That being said, I am interested in applying machine learning methods to a specific subset of my dataset. In social behavior, it is not only useful to quantify the behaviors exhibited by each individual but also the distance between them. For example, the distance between a mom and her calf can be indicative of the calves’ dependence on its mom (Nielsen et al., 2019). However, continuously measuring the distance between two individuals throughout a video is tedious and time intensive, so training a machine learning model could be an effective use of time. I plan to work with an intern this summer to develop a machine learning model to track the distance between pairs of gray whales in our drone footage and then relate this distance data with the manually coded behaviors to examine patterns in social behavior (Figure 4).  Stay tuned to learn more about our progress!

Figure 4. A mom and calf pair surfacing together. Image collected under NOAA/NMFS permit #21678

Did you enjoy this blog? Want to learn more about marine life, research, and conservation? Subscribe to our blog and get a weekly alert when we make a new post! Just add your name into the subscribe box on the left panel.  

References

Anderson, D. J., & Perona, P. (2014). Toward a Science of Computational Ethology. Neuron, 84(1), 18–31. https://doi.org/10.1016/j.neuron.2014.09.005

Bozek, K., Hebert, L., Portugal, Y., Mikheyev, A. S., & Stephens, G. J. (2021). Markerless tracking of an entire honey bee colony. Nature Communications, 12(1), 1733. https://doi.org/10.1038/s41467-021-21769-1

Ebbesen, C. L., & Froemke, R. C. (2021). Body language signals for rodent social communication. Current Opinion in Neurobiology, 68, 91–106. https://doi.org/10.1016/j.conb.2021.01.008

Mayo, C. A., & Marx, M. K. (1990). Surface foraging behaviour of the North Atlantic right whale, Eubalaena glacialis , and associated zooplankton characteristics. Canadian Journal of Zoology, 68(10), 2214–2220. https://doi.org/10.1139/z90-308

Mearns, D. S., Donovan, J. C., Fernandes, A. M., Semmelhack, J. L., & Baier, H. (2020). Deconstructing Hunting Behavior Reveals a Tightly Coupled Stimulus-Response Loop. Current Biology, 30(1), 54-69.e9. https://doi.org/10.1016/j.cub.2019.11.022

Nielsen, M., Sprogis, K., Bejder, L., Madsen, P., & Christiansen, F. (2019). Behavioural development in southern right whale calves. Marine Ecology Progress Series, 629, 219–234. https://doi.org/10.3354/meps13125

Pereira, T. D., Shaevitz, J. W., & Murthy, M. (2020). Quantifying behavior to understand the brain. Nature Neuroscience, 23(12), 1537–1549. https://doi.org/10.1038/s41593-020-00734-z

Of snakes and whales: How food availability and body condition affect reproduction

Clara Bird, PhD Student, OSU Department of Fisheries, Wildlife, and Conservation Sciences, Geospatial Ecology of Marine Megafauna Lab

Over six field seasons the GEMM lab team has conducted nearly 500 drone flights over gray whales, equaling over 100 hours of footage. These hours of footage are the central dataset for my PhD dissertation, so it’s up to me to process them all. This process can be challenging, tedious, and daunting, but it is also quite fun and a privilege to be the one person who gets to watch all the footage. It’s fascinating to get to know the whales and their behaviors and pick up on patterns. It motivates me to get through this video processing step and start doing the data analysis. Recently, it’s been especially fun to notice patterns that I’ve seen mentioned in the literature. One example is adult social behavior. 

There are two categories of social behavior that I’m interested in studying: maternal behavior, defined as interactions between a mom and its calf, and general social behaviors, defined as social interactions between non-mom/calf pairs. In this blog I’ll focus on general social behaviors, but if you’re interested in maternal behavior check out this blog. General social behavior, which I’ll refer to as social behavior moving forward, includes tactile interactions and promiscuous behaviors (Torres et al. 2018; Clip 1). While gray whales in the PCFG range are primarily foraging, researchers have observed increases in social behavior towards the end of the foraging season (Stelle et al., 2008; Torres et al., 2018). We think that this indicates that the whales are starting to focus less on feeding and more on breeding. This tradeoff of foraging vs. socializing time is interesting because it comes at an energetic cost.

Clip 1. Example of social interaction between a male and female gray whale off the coast of Oregon, USA. Collected under NOAA/NMFS permit #21678

Broadly, animals need to balance the energetic demands of survival with those of reproduction. They need to reproduce to pass on their genes, but reproduction is energetically demanding, and animals also need to survive and grow to be able to reproduce. The decision to reproduce is costly because reproduction requires energetic investment and time investment since animals do not forage (gaining energy) when they are socializing. Consequently, only animals with sufficient energy reserves (i.e., body condition) to invest in reproduction actually engage in reproduction. Given these costs associated with reproduction, we expect to see a relationship between social behavior and body condition (Green, 2001) with mainly animals in good body condition engaging in social behavior because these animals have sufficient reserves to sustain the cost. Furthermore, since body condition is an indicator of foraging success and prey availability, environmental conditions can also affect social behavior and reproduction through this pathway. 

Rahman et al. (2014) used a lab experiment to study the relationship between nutritional stress and male guppy courtship behavior (Figure 1). In their experiment they tested for the effects of both decreased diet quantity and quality on the frequency of male courtship behaviors. Rahman et al (2014) found that individuals in the low-quantity group were significantly smaller than those in the high-quality group and that diet quantity had a significant effect on the frequency of courtship behaviors. Males fed a low-quantity diet performed fewer courtship behaviors. Interestingly, there was no significant effect of diet quality on courtships behavior, although there was some evidence of an interaction effect, which suggests that within the low-quantity group, males fed with high-quality food performed more courtship behaviors that those fed with low-quality food. This study is interesting because it shows how foraging success (diet quantity and quality) can affect courting behavior. 

Figure 1. A guppy (Rahman et al., 2013)

However, guppies are not the ideal species for comparison to gray whales because gray whales and guppies have quite different life history traits. A more fitting comparison would be with an example species with more in common with gray whales, such as viviparous capital breeders. Viviparous animals develop the embryo inside the body and give live birth. Capital breeders forage to build energy reserves and then rely on those energy reserves during reproduction. Surprisingly, I found asp vipers to be a good example species for comparison to gray whales.

Asp vipers (Figure 2) are viviparous snakes who are considered capital breeders because they forage prior to hibernation, and then begin reproduction immediately following hibernation without additional foraging. Naulleau & Bonnet (1996) conducted a field study on female asp vipers to determine if there was a difference in body condition at the start of the breeding season between females who reproduced or not during that season. To do this they marked individuals and measured their body condition at the start of the breeding season and then recaptured those individuals at the end of the breeding season and recorded whether the individual had reproduced. Interestingly, they found that there was a strongly significant difference in body condition between females that did and did not reproduce. In fact, they discovered that no female below a certain body condition value reproduced, meaning that they found a body condition threshold for reproduction. 

Figure 2. An asp viper

Additionally, a study on water pythons found that their body condition threshold for reproduction shifted over time in response to prey availability (Madsen & Shine, 1999). These authors found that females lowered their threshold after several consecutive years of poor prey availability. These studies are really exciting to me because they address questions that the GRANITE project team is interested in tackling.

Understanding the relationship between body condition and reproduction in gray whales is an important puzzle piece for our work. The aim of the GRANITE project is to understand how the effects of stressors on individual whales scales up to population level impacts (read Lisa’s blog to learn more). Reproduction rates play a big role in population dynamics, so it is important to understand what factors affect reproduction. Since we’re studying these whales on their foraging grounds, assessing body condition provides an important link between foraging behavior and reproduction. 

For example, if an individual’s response to a stressor is to forage less, that may lead to poorer body condition, meaning that they may be less likely to reproduce. While reduced reproduction in one individual may not have a big effect on the population, the same response from multiple individuals could impact the population’s dynamics (i.e., increasing or decreasing abundance). Understanding these different relationships between behavior, body condition, and reproduction rates is a big undertaking, but it’s exciting to be a member of the GRANITE team as this strong group of scientists works to bring together different data streams to work on this big picture question. We’re all deep into data processing right now so stay tuned over the next few years to learn more about gray whale social behavior and to find out if fat whales are more social than skinny whales. 

Did you enjoy this blog? Want to learn more about marine life, research, and conservation? Subscribe to our blog and get weekly updates and more! Just add your name into the subscribe box on the left panel.  

References

Green, A. J. (2001). Mass/Length Residuals: Measures of Body Condition or Generators of Spurious Results? Ecology, 82(5), 1473–1483. https://doi.org/10.1890/0012-9658(2001)082[1473:MLRMOB]2.0.CO;2

Madsen, T., & Shine, R. (1999). The adjustment of reproductive threshold to prey abundance in a capital breeder. Journal of Animal Ecology, 68(3), 571–580. https://doi.org/10.1046/j.1365-2656.1999.00306.x

Naulleau, G., & Bonnet, X. (1996). Body Condition Threshold for Breeding in a Viviparous Snake. Oecologia, 107(3), 301–306.

Rahman, M. M., Kelley, J. L., & Evans, J. P. (2013). Condition-dependent expression of pre- and postcopulatory sexual traits in guppies. Ecology and Evolution, 3(7), 2197–2213. https://doi.org/10.1002/ece3.632

Rahman, M. M., Turchini, G. M., Gasparini, C., Norambuena, F., & Evans, J. P. (2014). The Expression of Pre- and Postcopulatory Sexually Selected Traits Reflects Levels of Dietary Stress in Guppies. PLOS ONE, 9(8), e105856. https://doi.org/10.1371/journal.pone.0105856

Stelle, L. L., Megill, W. M., & Kinzel, M. R. (2008). Activity budget and diving behavior of gray whales (Eschrichtius robustus) in feeding grounds off coastal British Columbia. Marine Mammal Science, 24(3), 462–478. https://doi.org/10.1111/j.1748-7692.2008.00205.x

Torres, L. G., Nieukirk, S. L., Lemos, L., & Chandler, T. E. (2018). Drone up! Quantifying whale behavior from a new perspective improves observational capacity. Frontiers in Marine Science, 5(SEP). https://doi.org/10.3389/fmars.2018.00319