Identifying birdsong computationally
June 3, 2008 3:33 AM Subscribe
Is it feasible to identify different birdsongs computationally?
So I've recently moved to a house with a big garden, and have become interested in identifying the different types of birds that visit. I wondered, since different bird songs are so well known, whether one could computationally identify birds by their songs. I was thinking along the lines of
1) stick a microphone in the garden and record a few hours of audio
2) some sort of preprocessing step; work out what frequencies are dominated by background noise, maybe cluster the individual songs in some way (since most will be repeated many times during the recording period)
3) use a neural network or such to assign a probability score for each bird (maybe take into account the location & time of year, in a Bayesian scheme)
Possible complication: if there are a lot of birds singing at the same time, their songs may always overlap - but still, given technology like this it seems like it should be possible. Or maybe you could use two mikes, and use the phase shift in the recording to distinguish songs coming from different places?
So I've recently moved to a house with a big garden, and have become interested in identifying the different types of birds that visit. I wondered, since different bird songs are so well known, whether one could computationally identify birds by their songs. I was thinking along the lines of
1) stick a microphone in the garden and record a few hours of audio
2) some sort of preprocessing step; work out what frequencies are dominated by background noise, maybe cluster the individual songs in some way (since most will be repeated many times during the recording period)
3) use a neural network or such to assign a probability score for each bird (maybe take into account the location & time of year, in a Bayesian scheme)
Possible complication: if there are a lot of birds singing at the same time, their songs may always overlap - but still, given technology like this it seems like it should be possible. Or maybe you could use two mikes, and use the phase shift in the recording to distinguish songs coming from different places?
Best answer: Well, there's tons of software to look at the acoustics of language. I used it for my master's thesis. I'm sure it wouldn't be a stretch to do spectographic analysis of bird song.
Once you get a hold of the software (and here's a free one, though I've not used it), you can get examples of bird song from a CD or even here.
Apparently Cornell also has a place to download not just song but also spectograms of the song, using a program called Raven Viewer that they offer on that page as well.
So, anyhoo, as to the process of writing some kind of program, well, I can't help you out there. But at least I've tried to identify tools to help you out with the background.
posted by Stewriffic at 3:53 AM on June 3, 2008
Once you get a hold of the software (and here's a free one, though I've not used it), you can get examples of bird song from a CD or even here.
Apparently Cornell also has a place to download not just song but also spectograms of the song, using a program called Raven Viewer that they offer on that page as well.
So, anyhoo, as to the process of writing some kind of program, well, I can't help you out there. But at least I've tried to identify tools to help you out with the background.
posted by Stewriffic at 3:53 AM on June 3, 2008
Oh, and even with a few birds at once, since a spectogram measures amplitude, you can also just look to see which song is the loudest.
But really, just go birding with an iPod loaded with songs. If you learn to ID both by eye and by ear over time, that will help you be a better birder in general.
Also something to take into account is that birds have many, MANY different sounds that they make, from alarm calls, to general song, to mimicking other birds, so you'll have to take that into account. I'd find out which birds are the most common in your area first, because you'll be dealing with a LOT of data even with just a small set of birds.
The Carolina Wren, for example, usually makes this sound. But they also do this one and this scolding sound. Here's a neat clip that has two Carolina Wrens, one singing the typical call and the other hissing in the background. So, yeah, lots of data.
I'm SO interested in seeing what you might come up with! As you might tell, I'm an avid birder with some background in acoustic analysis.
posted by Stewriffic at 4:03 AM on June 3, 2008
But really, just go birding with an iPod loaded with songs. If you learn to ID both by eye and by ear over time, that will help you be a better birder in general.
Also something to take into account is that birds have many, MANY different sounds that they make, from alarm calls, to general song, to mimicking other birds, so you'll have to take that into account. I'd find out which birds are the most common in your area first, because you'll be dealing with a LOT of data even with just a small set of birds.
The Carolina Wren, for example, usually makes this sound. But they also do this one and this scolding sound. Here's a neat clip that has two Carolina Wrens, one singing the typical call and the other hissing in the background. So, yeah, lots of data.
I'm SO interested in seeing what you might come up with! As you might tell, I'm an avid birder with some background in acoustic analysis.
posted by Stewriffic at 4:03 AM on June 3, 2008
How you would deal with the many birds that mimic the sounds of other birds is another interesting complication.
posted by le morte de bea arthur at 4:05 AM on June 3, 2008
posted by le morte de bea arthur at 4:05 AM on June 3, 2008
Best answer: That sounds like a great experiment and I'd love to hear about how it goes. A suggestion: we have a lot of birds and they tend to get a particular spot where they like to sing, and a particular time of day, so I would imagine that you could make use of this information for your analysis as it emerges, probably not weighted as strongly as other factors. Some other ways that birdsongs are distinguished, other than frequency range:
length of song
number of permutations (i.e. how many "blocks" the song is made up of)
distance between highest and lowest tone, largest distance in a single interval
time that "notes" are held
The 'distorted' punctuation marks that many birds break up their songs with probably look very distinctive using spectography.
Here is a non-free research paper that could help you:
http://ieeexplore.ieee.org/Xplore/login.jsp?url=/iel5/9248/29347/01327238.pdf?arnumber=1327238
posted by Your Time Machine Sucks at 4:18 AM on June 3, 2008
length of song
number of permutations (i.e. how many "blocks" the song is made up of)
distance between highest and lowest tone, largest distance in a single interval
time that "notes" are held
The 'distorted' punctuation marks that many birds break up their songs with probably look very distinctive using spectography.
Here is a non-free research paper that could help you:
http://ieeexplore.ieee.org/Xplore/login.jsp?url=/iel5/9248/29347/01327238.pdf?arnumber=1327238
posted by Your Time Machine Sucks at 4:18 AM on June 3, 2008
Another one is how abrupt a tone change is. Some birds change tones in a way which probably resembles flute legato, others will probably look more like a staircase, with nothing in the way of transitional tones over time. I'm listening to a blackbird right now and there is no perceptible elision or tone "bending", just quick groups of distinct tones.
Hilariously/disturbingly, I'm fairly sure that it has started to include an imitation of an ambulance in there -- there is now often a funny little alternating tritone at the end of certain phrases.
posted by Your Time Machine Sucks at 5:02 AM on June 3, 2008
Hilariously/disturbingly, I'm fairly sure that it has started to include an imitation of an ambulance in there -- there is now often a funny little alternating tritone at the end of certain phrases.
posted by Your Time Machine Sucks at 5:02 AM on June 3, 2008
Best answer: Here is a non-free research paper that could help you:
http://ieeexplore.ieee.org/Xplore/login.jsp?url=/iel5/9248/29347/01327238.pdf?arnumber=1327238
free version here
The problem with the high-level features people have suggested here is that they are really hard to compute reliably. Maybe you could try to apply some techniques from automatic speaker recognition to bird songs, i.e. you could covert recordings to a parametric representation (MFCCs are popular), estimate GMMs from the data (using e.g. Expectation Maximization) and compare the likelihoods of MFCC vectors extracted from different samples given a GMM.
I'm not sure that would work though, these low-level features are really sensitive to noise and variation so unless you manage to lure your birds into a sound-proof recording studio, this will be a problem. You also need quite a lot of data to compute the GMMs, so maybe Vector Quantization on the MFCCs would be the way to go, there are some Matlab scripts for that here ("Les scripts").
But of course I have no idea whether any of that would work at all since inter-speaker variation and inter-bird species variation aren't exactly the same thing.
posted by snownoid at 6:21 AM on June 3, 2008
http://ieeexplore.ieee.org/Xplore/login.jsp?url=/iel5/9248/29347/01327238.pdf?arnumber=1327238
free version here
The problem with the high-level features people have suggested here is that they are really hard to compute reliably. Maybe you could try to apply some techniques from automatic speaker recognition to bird songs, i.e. you could covert recordings to a parametric representation (MFCCs are popular), estimate GMMs from the data (using e.g. Expectation Maximization) and compare the likelihoods of MFCC vectors extracted from different samples given a GMM.
I'm not sure that would work though, these low-level features are really sensitive to noise and variation so unless you manage to lure your birds into a sound-proof recording studio, this will be a problem. You also need quite a lot of data to compute the GMMs, so maybe Vector Quantization on the MFCCs would be the way to go, there are some Matlab scripts for that here ("Les scripts").
But of course I have no idea whether any of that would work at all since inter-speaker variation and inter-bird species variation aren't exactly the same thing.
posted by snownoid at 6:21 AM on June 3, 2008
Best answer: I worked with a researcher who was using omnidirectional microphones to record bird songs in the boreal forest for later identification (by bird song experts) which sounds like what you want to do (but with a computer for later identification).
(Sorry these are not public access)
Rempel et al. 2005. Bioacoustic monitoring of forest songbirds: interpreter variability and effects of configuration and digital processing methods in the laboratory. J Field Ornith 76:1-11.
Hobson et al. 2002. Acoustic surveys of birds using electronic recordings: new potential from an omnidirectional microphone system. Wild Soc Bull. 30: 709-720.
Anyway, if you have access to Web of Science, you could check out the articles that cite these two and see what developments have been made.
Also, the Birds of North America series based out of Cornell (again not totally public access) has recordings for most birds in North America, including their spectrographs (? those little black and white images that show song pitch and duration). Actually, check out the Cornell site for a bunch of animal sound related stuff. It looks like they have the analysis tools there (don't know how available to the public).
If you don't have access to Web of Science, or something like that, message me and I'll see what a little more digging can find (and now back to my own literature review).
posted by hydrobatidae at 8:37 AM on June 3, 2008
(Sorry these are not public access)
Rempel et al. 2005. Bioacoustic monitoring of forest songbirds: interpreter variability and effects of configuration and digital processing methods in the laboratory. J Field Ornith 76:1-11.
Hobson et al. 2002. Acoustic surveys of birds using electronic recordings: new potential from an omnidirectional microphone system. Wild Soc Bull. 30: 709-720.
Anyway, if you have access to Web of Science, you could check out the articles that cite these two and see what developments have been made.
Also, the Birds of North America series based out of Cornell (again not totally public access) has recordings for most birds in North America, including their spectrographs (? those little black and white images that show song pitch and duration). Actually, check out the Cornell site for a bunch of animal sound related stuff. It looks like they have the analysis tools there (don't know how available to the public).
If you don't have access to Web of Science, or something like that, message me and I'll see what a little more digging can find (and now back to my own literature review).
posted by hydrobatidae at 8:37 AM on June 3, 2008
Best answer: At MPOW, we have a researcher who is looking for the infamous Ivory Billed Woodpecker by making 24 hour field recordings and then analyzing the results for evidence of its song. His team used manual scanning of the resulting song spectrograms but also scanned the recordings using Syrinx-PC Sound Analysis Software. Dr. Mennill recommends other software as well for bird song analysis.
It would appear that no one has developed a neural network to distinguish bird songs automagically. Good luck!
posted by copystar at 3:56 AM on June 4, 2008
It would appear that no one has developed a neural network to distinguish bird songs automagically. Good luck!
posted by copystar at 3:56 AM on June 4, 2008
Response by poster: Ahh, would that I had the time to embark upon such a project! But I'm afraid it'll have to stay at the "idle curiosity" stage.
posted by primer_dimer at 8:11 AM on June 4, 2008
posted by primer_dimer at 8:11 AM on June 4, 2008
yeah...it's a pretty big bite to chew.
posted by Stewriffic at 5:55 PM on June 5, 2008
posted by Stewriffic at 5:55 PM on June 5, 2008
This thread is closed to new comments.
Oh man, how cool am I for having 3-year-old emails: The investigator was Tiffany Bloomfield at the University of Chicago. There is no direct contact information in the email, but perhaps you could look her up and see what it was about.
posted by phunniemee at 3:47 AM on June 3, 2008