All Thresholds Barred: Direct Estimation of Call Density in Bioacoustic Data

• Main Authors: Navine, Amanda K, Denton, Tom, Weldy, Matthew J, Hart, Patrick J
• Format: Journal Article
• Language: English
• Published: 23-02-2024
• Subjects: Computer Science - Learning; Computer Science - Sound; Quantitative Biology - Quantitative Methods
• DOI: 10.48550/arxiv.2402.15360

Passive acoustic monitoring (PAM) studies generate thousands of hours of
audio, which may be used to monitor specific animal populations, conduct broad
biodiversity surveys, detect threats such as poachers, and more. Machine
learning classifiers for species identification are increasingly being used to
process the vast amount of audio generated by bioacoustic surveys, expediting
analysis and increasing the utility of PAM as a management tool. In common
practice, a threshold is applied to classifier output scores, and scores above
the threshold are aggregated into a detection count. The choice of threshold
produces biased counts of vocalizations, which are subject to false
positive/negative rates that may vary across subsets of the dataset. In this
work, we advocate for directly estimating call density: The proportion of
detection windows containing the target vocalization, regardless of classifier
score. Our approach targets a desirable ecological estimator and provides a
more rigorous grounding for identifying the core problems caused by
distribution shifts -- when the defining characteristics of the data
distribution change -- and designing strategies to mitigate them. We propose a
validation scheme for estimating call density in a body of data and obtain,
through Bayesian reasoning, probability distributions of confidence scores for
both the positive and negative classes. We use these distributions to predict
site-level densities, which may be subject to distribution shifts. We test our
proposed methods on a real-world study of Hawaiian birds and provide simulation
results leveraging existing fully annotated datasets, demonstrating robustness
to variations in call density and classifier model quality.