Population‐Level Acoustic Classification of Salish Sea Killer Whales: Integrating Biologically Informed Call Type Balancing to Build Robust Models for Conservation Monitoring

Abstract

There is a pressing need to build population‐specific acoustic classifiers for killer whales ( Orcinus orca ) in the Salish Sea. However, building datasets that result in generalizable models is challenging due to diverse killer whale repertoires and confounding signals such as humpback whale calls and environmental noise. In this work we focus on building biologically informed training datasets with expert guided call type balancing and measure its effects on detection and classification performance. Neural network models were created using the BirdNET framework and were trained to recognize Southern Resident (SRKW) and West Coast Transient (TKW) killer whales, alongside humpback whales and background noise. Models were trained on nine systematically constructed datasets (12,000 samples each) that varied the percentage of call type labeled replacements (0%, 10%, 30%) per killer whale population while holding humpback and background annotations constant. Models were evaluated on 20% of the available training data (holdout) and a novel dataset from a different recording setup and location. Performance was assessed with precision–recall, area under the precision‐recall curve, mean average precision, and Matthews Correlation Coefficient to measure misclassification. For both killer whale classes, performance on the in‐sample data was high with AUC values above 0.873 but degraded when applied to data collected from a different hydrophone system. Detection and classification accuracy for SRKW was consistently high, while the ability of the models to generalize performance to out‐of‐distribution data was strongly dependent on call type balancing. Model generalization was sensitive to how call type labeled data were incorporated into the training datasets, with different inclusion strategies producing distinct generalization behaviors across populations. These results demonstrate the value of incorporating domain knowledge when training bioacoustics neural networks and highlight the need for publicly available call type catalogs across populations.

Bibliographic Information