Training a computer to recognise bird calls turns out to be a thorny mathematical challenge.
Mathematician Stephen Marsland and Isabel Castro - pictured with Blandy the kiwi - are collaborating on a project to enable computers to recognise bird calls. Photo: Supplied
Find Our Changing World on Apple Podcasts, Spotify, Stitcher, iHeartRADIO, Google Podcasts, RadioPublic or wherever you listen to your podcasts
Imagine you are in a forest in New Zealand. It’s night. A light wind is rustling through the canopy, and it is raining. Off to the right you can hear the shrill hunting trill of a ruru.
In the distance a pair of North Island brown kiwi are duetting – the higher whistle of the male is answered by the low growl of the female.
You are using sound to identify which birds are calling and you can easily make out the direction the call is coming from. Without you even noticing your brain subtracts the white noise of the wind and rain so you concentrate on the important cues.
Now imagine you are back in your office, you have an acoustic recording of the same night forest and you want to analyse it to find out what species are present and how any there were. How can you do it without having to listen to the entire recording?
Because it turns out that there isn’t a computer programme that can do what we humans can do.
And the avian equivalent of the Shazam app that lets you identify any song from just a few seconds of audio doesn’t exist. Yet.
But it’s something that Massey University ecologist Isabel Castro and Victoria University of Wellington mathematician Stephen Marsland would like to create.
A spectrogram is the visual representation of the frequencies in a bird call, in this case a male North Island brown kiwi recorded at very close quarters. Photo: Stephen Marsland / VUW
Isabel and Stephen are now the first to admit that if automated birdsong analysis was easy someone would have already done it, and after more than six years of work they are only just making progress.
Stephen says that computer recognition of sound uses pictures. The pictures are visual representations of the bird’s calls, spectrograms or sonograms which show time on the horizontal axis and frequency on the vertical axis.
What Stephen and a group of students have been doing is finding mathematical ways of describing the shape of a bird call that a computer can use as its ‘search image’.
A spectrogram of the duet calls of a pair of North Island brown kiwi. The higher frequency calls for the first 30 seconds are the male (with a layer of harmonics) and the lower-pitched female joins in halfway through. Photo: Stephen Marsland / VUW
“But what we’ve discovered is how hard the real world is,” says Stephen. The bush is a very noisy place and a faint bird call in the distance is easily obscured by an amorphous wide frequency band of wind and rain.
The AviaNZ project is developing birdsong analysis software. To date Stephen has worked on kiwi, ruru, kakapo and bittern.
AviaNZ is an open source project with a website where people are encouraged to add filters for new New Zealand species that other researchers can then use.
A spectrogram of ruru calls. The first two small calls in the bottom left are the familiar ruru calls, while the stacks of white are a hunting trill, featuring layers of harmonics. Photo: Stephen Marsland / VUW
Listen to the full podcast to hear Isabel and Stephen talk about the AviaNZ project.
