Identifying birdsong computationally

June 3, 2008 3:33 AM

Is it feasible to identify different birdsongs computationally?

So I've recently moved to a house with a big garden, and have become interested in identifying the different types of birds that visit. I wondered, since different bird songs are so well known, whether one could computationally identify birds by their songs. I was thinking along the lines of

1) stick a microphone in the garden and record a few hours of audio
2) some sort of preprocessing step; work out what frequencies are dominated by background noise, maybe cluster the individual songs in some way (since most will be repeated many times during the recording period)
3) use a neural network or such to assign a probability score for each bird (maybe take into account the location & time of year, in a Bayesian scheme)

Possible complication: if there are a lot of birds singing at the same time, their songs may always overlap - but still, given technology like this it seems like it should be possible. Or maybe you could use two mikes, and use the phase shift in the recording to distinguish songs coming from different places?

posted by primer_dimer to computers & internet (11 comments total)

This is unhelpful, but I participated in a research study a while ago where I was asked to distinguish between the songs of different kinds of birds by recognizing patterns in them. I vaguely recall it having something to do with a program they were trying to write, though it ostensibly could have been to see if humans have similar speech patterns.

Oh man, how cool am I for having 3-year-old emails:

Hello,

We are now signing up additional times for a study on sound classification. If you have not yet participated in this experiment, we would like to offer you the opportunity to participate this week and earn $10 for 30 minutes of your time.

To be eligible for this experiment, you must be 18 or older, right-handed, and a native speaker of American English. You must also have normal or corrected-to-normal vision and hearing. If you need glasses or contacts, you must have them with you on the day of the experiment.

The investigator was Tiffany Bloomfield at the University of Chicago. There is no direct contact information in the email, but perhaps you could look her up and see what it was about.
posted by phunniemee at 3:47 AM on June 3

Well, there's tons of software to look at the acoustics of language. I used it for my master's thesis. I'm sure it wouldn't be a stretch to do spectographic analysis of bird song.

Once you get a hold of the software (and here's a free one, though I've not used it), you can get examples of bird song from a CD or even here.

Apparently Cornell also has a place to download not just song but also spectograms of the song, using a program called Raven Viewer that they offer on that page as well.

So, anyhoo, as to the process of writing some kind of program, well, I can't help you out there. But at least I've tried to identify tools to help you out with the background.
posted by Stewriffic at 3:53 AM on June 3

Oh, and even with a few birds at once, since a spectogram measures amplitude, you can also just look to see which song is the loudest.

But really, just go birding with an iPod loaded with songs. If you learn to ID both by eye and by ear over time, that will help you be a better birder in general.

Also something to take into account is that birds have many, MANY different sounds that they make, from alarm calls, to general song, to mimicking other birds, so you'll have to take that into account. I'd find out which birds are the most common in your area first, because you'll be dealing with a LOT of data even with just a small set of birds.

The Carolina Wren, for example, usually makes this sound. But they also do this one and this scolding sound. Here's a neat clip that has two Carolina Wrens, one singing the typical call and the other hissing in the background. So, yeah, lots of data.

I'm SO interested in seeing what you might come up with! As you might tell, I'm an avid birder with some background in acoustic analysis.
posted by Stewriffic at 4:03 AM on June 3

How you would deal with the many birds that mimic the sounds of other birds is another interesting complication.
posted by le morte de bea arthur at 4:05 AM on June 3

That sounds like a great experiment and I'd love to hear about how it goes. A suggestion: we have a lot of birds and they tend to get a particular spot where they like to sing, and a particular time of day, so I would imagine that you could make use of this information for your analysis as it emerges, probably not weighted as strongly as other factors. Some other ways that birdsongs are distinguished, other than frequency range:

length of song
number of permutations (i.e. how many "blocks" the song is made up of)
distance between highest and lowest tone, largest distance in a single interval
time that "notes" are held
The 'distorted' punctuation marks that many birds break up their songs with probably look very distinctive using spectography.

Here is a non-free research paper that could help you:

http://ieeexplore.ieee.org/Xplore/login.jsp?url=/iel5/9248/29347/01327238.pdf?arnumber=1327238
posted by Your Time Machine Sucks at 4:18 AM on June 3

Another one is how abrupt a tone change is. Some birds change tones in a way which probably resembles flute legato, others will probably look more like a staircase, with nothing in the way of transitional tones over time. I'm listening to a blackbird right now and there is no perceptible elision or tone "bending", just quick groups of distinct tones.

Hilariously/disturbingly, I'm fairly sure that it has started to include an imitation of an ambulance in there -- there is now often a funny little alternating tritone at the end of certain phrases.
posted by Your Time Machine Sucks at 5:02 AM on June 3

Here is a non-free research paper that could help you:

http://ieeexplore.ieee.org/Xplore/login.jsp?url=/iel5/9248/29347/01327238.pdf?arnumber=1327238

free version here

The problem with the high-level features people have suggested here is that they are really hard to compute reliably. Maybe you could try to apply some techniques from automatic speaker recognition to bird songs, i.e. you could covert recordings to a parametric representation (MFCCs are popular), estimate GMMs from the data (using e.g. Expectation Maximization) and compare the likelihoods of MFCC vectors extracted from different samples given a GMM.
I'm not sure that would work though, these low-level features are really sensitive to noise and variation so unless you manage to lure your birds into a sound-proof recording studio, this will be a problem. You also need quite a lot of data to compute the GMMs, so maybe Vector Quantization on the MFCCs would be the way to go, there are some Matlab scripts for that here ("Les scripts").
But of course I have no idea whether any of that would work at all since inter-speaker variation and inter-bird species variation aren't exactly the same thing.
posted by snownoid at 6:21 AM on June 3

I worked with a researcher who was using omnidirectional microphones to record bird songs in the boreal forest for later identification (by bird song experts) which sounds like what you want to do (but with a computer for later identification).

(Sorry these are not public access)

Rempel et al. 2005. Bioacoustic monitoring of forest songbirds: interpreter variability and effects of configuration and digital processing methods in the laboratory. J Field Ornith 76:1-11.

Hobson et al. 2002. Acoustic surveys of birds using electronic recordings: new potential from an omnidirectional microphone system. Wild Soc Bull. 30: 709-720.

Anyway, if you have access to Web of Science, you could check out the articles that cite these two and see what developments have been made.

Also, the Birds of North America series based out of Cornell (again not totally public access) has recordings for most birds in North America, including their spectrographs (? those little black and white images that show song pitch and duration). Actually, check out the Cornell site for a bunch of animal sound related stuff. It looks like they have the analysis tools there (don't know how available to the public).

If you don't have access to Web of Science, or something like that, message me and I'll see what a little more digging can find (and now back to my own literature review).
posted by hydrobatidae at 8:37 AM on June 3

At MPOW, we have a researcher who is looking for the infamous Ivory Billed Woodpecker by making 24 hour field recordings and then analyzing the results for evidence of its song. His team used manual scanning of the resulting song spectrograms but also scanned the recordings using Syrinx-PC Sound Analysis Software. Dr. Mennill recommends other software as well for bird song analysis.

It would appear that no one has developed a neural network to distinguish bird songs automagically. Good luck!
posted by copystar at 3:56 AM on June 4

Ahh, would that I had the time to embark upon such a project! But I'm afraid it'll have to stay at the "idle curiosity" stage.
posted by primer_dimer at 8:11 AM on June 4

yeah...it's a pretty big bite to chew.
posted by Stewriffic at 5:55 PM on June 5