8 ms·

Bird song sonographs show distinct drawing patterns

195 points by algui91 5y ago

DoingIsLearning 5y ago
Dupe: https://news.ycombinator.com/item?id=27030171 https://news.ycombinator.com/item?id=27030171
- algui91 5y ago
  Ups, I though if some dup is detected I would be warned before submitting. Will remove it. Thanks! Edit: Seems I can not remove it by myself :-(
  - neogodless 5y ago
    No worries - the previous entry is not front page, and I missed it. Glad I caught it the second time around.
slver 5y ago
I choose to imagine they’re API endpoints communicating lists of entities with similar data causing the patterns.
debbiedowner 5y ago
HSV color scheme looks beautiful, but since it doesn't seem to have perceptually uniform color change we won't see it used in papers much :( I think the rule for audio papers is linear change in color according to energy? So this rep may lead us to hallucinate our conclusions?
throwamon 5y ago
For a moment I thought they would hypothesize that the patterns might be an actual “rendering” of what the birds were looking at, especially when I saw the 7th image which looks remarkably similar to a rodent. Apparently I'm pretty far off, but just this idea of animals communicating images with sound waves was something I'd otherwise never have.
- jerrre 5y ago
  Interesting train of thought. Perhaps it's good to know that the x axis (from left to right) is temporal. So for someone/something to translate this to an image with some special sense would also necessitate some memory part. With a big fantasy you could continue your hypothesis by comparing it to describing an object you see with words by telling the height of the outline from left to right.
- kurthr 5y ago
  What if groups/species of birds had "collective synesthesia"? Some people see colors or feel tastes... and it doesn't seem that unlikely that there would be selection pressure for parts of the bird brain to connect their spatial awareness to their auditory system and then songs. There could be a nice feedback loop that their own songs would strike similar experiences for themselves (if they hear themselves similar to others). Of course we know that bats have this type of neural connection in echolocation and dolphins/whales may even using it to communicate in similar ways with their songs.
  - mattkrause 5y ago
    They kinda do! If you place migratory birds in a big round cage, they show 'Zugunruhe[0]', or migratory restlessness: they jump and flutter in the direction that they would be otherwise be migrating if they were out in the wild. Rotating the magnetic field (e.g., by putting magnets around the aviary) also rotates the direction of their Zugunruhe. No one totally understands how this works, but the magnetic information is thought to be 'overlaid' on other sensory information. One candidate pathway involves a light-dependent pathway in the retina. When a cryptochrome absorbs light, it generates radical pairs that affect how visual information is perceived[1]. This could give the birds something like HUD which displays magnetic field lines on top of the visual scene. Consistent with this idea, birds can only orient to magnetic fields under certain colors of light, with the color varying a bit from bird to bird[2]. It's almost as if the colored light washes out the HUD. There's another parallel pathway involving bits of magnetite in the beak[3,4]. These signals flow through the trigeminal nerve, which carries a lot of different signals; it would not be impossible for this to manifest as "pressure", as it carries touch/somatosensory information in many animals. [0] Ethologists love German and this word captures my lockdown feeling so very well. [1] https://www.sciencedirect.com/science/article/pii/S000634950076629X?via%3Dihub https://www.sciencedirect.com/science/article/pii/S000634950... [2] https://www.nature.com/articles/364525a0 https://www.nature.com/articles/364525a0 [3] https://www.sciencedirect.com/science/article/pii/S0006349507714108?via%3Dihub https://www.sciencedirect.com/science/article/pii/S000634950... [4] https://royalsocietypublishing.org/doi/10.1098/rspb.2009.0050 https://royalsocietypublishing.org/doi/10.1098/rspb.2009.005...
OmarShehata 5y ago
This is amazing. The GitHub repo has more information too, about how the common explanation of how our ears work (isolating sounds in an FFT-like-process) may be incorrect: > Contrary to what you might think, our ears don't seem to rely on an FFT-like process to extract isolated frequencies. Instead, our ears detect periodic parts in the signal, although in most cases those periodic parts closely match the FFT frequencies. There is a simple experiment that proves this point: https://github.com/soundshader/soundshader.github.io#why-acf-matches-our-perception-of-sound https://github.com/soundshader/soundshader.github.io#why-acf...
- jcims 5y ago
  This is my laymans test for the FFT theory. Put in earbuds or a headset and (quietly) play sounds that are slightly different in frequency (1-10hz). You will still hear a sort of ethereal beat tone between them that's different than the beat you would hear if you were listening to the same through speakers. I don't see how this would occur frequency-domain perception and it's too present (at least in my experiment) to be attributed to bone conduction across the skull.
  - morelisp 5y ago
    https://en.wikipedia.org/wiki/Beat_(acoustics)#Binaural_beats https://en.wikipedia.org/wiki/Beat_(acoustics)#Binaural_beat...
    - jcims 5y ago
      Whoa! At least that gives me some comfort I wasn't imagining it. I wish I understood some of the neurological explanation of how the beats are perceived because it doesn't seem to match with my understanding of the explanation of how our ears work. If everything is pushed into the frequency domain based on the stimulation of different parts of the cochlea, where does the time domain beat emerge?
      - filoeleven 5y ago
luxuryballs 5y ago
Snek! https://soundshader.github.io/hss/gallery/bird/3.png https://soundshader.github.io/hss/gallery/bird/3.png
temporalparts 5y ago
Slight related tangent: For all those interested in birding, I highly recommend the BirdNet app available on Android and iOS! You can record and send audio data, and it will try to classify the bird for you based on the recording. In the process of recording, it will show you the bird's spectrograms, which is really cool!
- algui91 5y ago
  That’s really cool! trying it out now. thanks for the recommendation.
topspin 5y ago
It makes sense to me that bird song would have few harmonics. Although we may appreciate the beauty of bird song they don't sing for aesthetics. They are signaling. Concentrating energy into a narrow band delivers greater range for the same energy than harmonically rich signals. We do likewise with electromagnetic communication.
- periheli0n 5y ago
  When a bird sings to attract a mating partner, competing against other birds, does it not sing for aesthetics? Also, couldn’t harmonics be used to improve decoding in noisy environments? Spreading the signal over a wide band is not unheard of in man-made electromagnetic communication either.
  - darig 5y ago
    Signal strength is the aesthetic. If you were a bird, you'd spend so much time trying to change what you were that you'd forget to eat your worms and die.
  - karmakaze 5y ago
    When using spread-spectrum transmission it would be counterproductive to utilize harmonics. Non-sinusoidal noise at frequency F would pollute frequencies k*F. Also the transmission band is likely less than an octave.
- enchiridion 5y ago
  I just had two thought considering this: 1. Do we appreciate bird songs because they indicate a resource rich/hospitable environment? 2. Do birds hear the entire range of the song, or just pick up specific bands?
- crazygringo 5y ago
  I think the explanation is even simpler. For a loud, high-pitched sound produced from a tiny object like a bird, is it even physically possible to have the loud resonances needed for overtones and timbre? I think it's just physics. Whales are "signaling" over huge distances too but have plenty of overtones, since that's what low frequencies in huge cavities produce. I'm not sure the dichotomy between "aesthetics" and "signaling" you suggest actually exists. A peacock's plumage is certainly aesthetics but also certainly signaling. And what is a birdsong's melody if not aesthetics?
grawprog 5y ago
Those are intersting. They remind me of the sonographs we collected of bat echolocation calls when we were doing bat surveys with anabat detectors. Just by the shape of the calls you could narrow the bat down to a likely species and determine whether it was just flying around, actively hunting or about to eat a hapless insect. It was a pretty awesome device.
wombatmobile 5y ago
Um, I'd like to be able to parse this article. but I don't know what it is saying. Can anyone explain the gist of it to me in simple terms? What does this mean? "These sonograms are remarkably different from other sounds, as if birds “draw” with sound something that’s flying backwards in time."
- noxToken 5y ago
  Andrew Huang does a good job of explaining harmonics and overtones[0]. You only need to watch until about the 4 minute mark (from the timestamp) to get an explanation of harmonics and what the sonograms represent. The short of it is that most natural sounds product a root tone plus a varying amount of related tones above it. Our ears hear the root tone, and the other tones above it are what give the sound its uniqueness. That's why a guitar, a clarinet and 3 singers can produce the same note while sounding distinct. Birds seem to produce a natural sound without a lot of the related tones above it. Their sound is, relatively speaking, much purer than most other natural sounds. That's very unique. [0]: https://youtu.be/Wx_kugSemfY?t=95 https://youtu.be/Wx_kugSemfY?t=95
- usrbin 5y ago
  Audio person here. I found the post fascinating, but I wish they did more to explain what they were talking about to a layperson. Basically, all sounds that you hear are composed of many layered sine waves of different frequencies and intensities. The graphs in the post are spectrograms, which graph those frequencies over time. The Y axis represents pitch, the X axis represents time, and the brightness at any given point represents how loud that particular frequency was at that particular time. Most sounds, even seemingly simple ones, look very complex on a sonogram, like a smudged pen stroke. The images of different instruments below demonstrate this; these are all very complex sounds, even though we only hear it as a single note being played. The voice one is one of my favorites, because it shows just how weird and complicated everyday sounds can get. But bird songs are different; on a sonogram, they appear as a single line. The complexity of the bird songs here comes from the fact that they're taking a single sine wave and changing the pitch over time. Where most sounds look like a complex mix of smudged paint strokes, bird songs look like a single, precise, bouncing stroke.
klyrs 5y ago
> I don’t know if it’s a feature or a defect of the instrument. It's a feature https://monoskop.org/images/a/af/Gough_C_2007_The_violin_Chladni_patterns_plates_shells_and_sounds.pdf https://monoskop.org/images/a/af/Gough_C_2007_The_violin_Chl...
kazinator 5y ago
> However it would be a mistake to to call flute sound simple: as you see, every level has its own regular pattern that can’t be recreated with a simple mix of sinusoidal tones. The resonance of the flute's air chamber is driven by the noise of forced air stream, which is why it contains high frequencies at all. Of course you can't recreate the flute sound just with some periodic sinusoidals, because the noise component has to be present. The noise is complex and is itself filtered by the flute's chamber, in a different way depending on which valves are closed. You can play a recognizeable scale on a flute without getting a tone out of it, just using air noise, the same way you can produce a musical tune using chhhh sounds out your mouth. Those notes, or something like them, are still there in the background when a tone is being produced. There is no flute tone without them.
gbh444g 5y ago
I wanted to add that the nature of those thin horizontal layers and the patterns they form is similar to diffraction. A laser beam passing thru a thin slit forms a pattern with the sinc-wave distribution of intensity, and so is FFT of a rectangular window used in these spectrograms makes a sinc-wave shape that "diffracts" the "true" sound frequency.
birdbrain 5y ago
I don't want to detract from the images, or the work that went into this, but, er, there are a lot of types of birds. Just among passerines (songbirds), we're talking over 6,000 species. So any sweeping statement like "birds don’t seem to bother to create a complex multi-layered harmonics pattern" is practically guaranteed to be wrong. And so it is. Lots and lots of birds sing incredibly harmonically complex songs. Browse any of these (https://www.remoteenvironmentalassessmentlaboratory.com/explore/birdsong/ https://www.remoteenvironmentalassessmentlaboratory.com/expl...) if you're interested - it's a tiny sample of birds, and many, many of them do in fact have harmonically complex songs.
- crazygringo 5y ago
  You may be misunderstanding, though your username is very appropriate. :) "Harmonics" here isn't referring to the harmonic complexity of songs. It's referring to harmonics as in overtones, the complexity of the overtones -- the timbre. It's the voice of the bird, not the songs it sings.
  - deleted 5y ago
    [deleted]
  - birdbrain 5y ago
    I should have been clearer. I am also talking about moment-to-moment spectral complexity or entropy. I agree that the point TFA is suggesting seems to be about the spectral complexity at any given moment in song - what you might call the timbre - and how nearly pure the tones in their spectrograms are. (They don't specify what species they're showing. Looks like several, but I can't ID them by eye.) My point is that no, in fact, many bird species produce vocalizations that are indeed spectrally complex (beyond even just harmonic stacks) from moment-to-moment. Take a look at song from a blue jay (https://www.remoteenvironmentalassessmentlaboratory.com/explore/birdsong/bluejay.php https://www.remoteenvironmentalassessmentlaboratory.com/expl... ; not the best example, or one I produced, but an easy one to hand), particularly the syllables near the end of the clip. That's an example of a complex timbre. And lots of species produce song and calls with features like this.
- 5y ago
humaniania 5y ago
Are there AI projects working on mapping bird song into possible meanings to figure out a language?

195 points by algui91 5y ago

DoingIsLearning 5y ago

Dupe: https://news.ycombinator.com/item?id=27030171 https://news.ycombinator.com/item?id=27030171

algui91 5y ago
Ups, I though if some dup is detected I would be warned before submitting. Will remove it. Thanks! Edit: Seems I can not remove it by myself :-(
- neogodless 5y ago
  No worries - the previous entry is not front page, and I missed it. Glad I caught it the second time around.

slver 5y ago

I choose to imagine they’re API endpoints communicating lists of entities with similar data causing the patterns.

debbiedowner 5y ago

HSV color scheme looks beautiful, but since it doesn't seem to have perceptually uniform color change we won't see it used in papers much :( I think the rule for audio papers is linear change in color according to energy? So this rep may lead us to hallucinate our conclusions?

throwamon 5y ago

For a moment I thought they would hypothesize that the patterns might be an actual “rendering” of what the birds were looking at, especially when I saw the 7th image which looks remarkably similar to a rodent. Apparently I'm pretty far off, but just this idea of animals communicating images with sound waves was something I'd otherwise never have.

jerrre 5y ago
Interesting train of thought. Perhaps it's good to know that the x axis (from left to right) is temporal. So for someone/something to translate this to an image with some special sense would also necessitate some memory part. With a big fantasy you could continue your hypothesis by comparing it to describing an object you see with words by telling the height of the outline from left to right.
kurthr 5y ago
What if groups/species of birds had "collective synesthesia"? Some people see colors or feel tastes... and it doesn't seem that unlikely that there would be selection pressure for parts of the bird brain to connect their spatial awareness to their auditory system and then songs. There could be a nice feedback loop that their own songs would strike similar experiences for themselves (if they hear themselves similar to others). Of course we know that bats have this type of neural connection in echolocation and dolphins/whales may even using it to communicate in similar ways with their songs.
- mattkrause 5y ago
  They kinda do! If you place migratory birds in a big round cage, they show 'Zugunruhe[0]', or migratory restlessness: they jump and flutter in the direction that they would be otherwise be migrating if they were out in the wild. Rotating the magnetic field (e.g., by putting magnets around the aviary) also rotates the direction of their Zugunruhe. No one totally understands how this works, but the magnetic information is thought to be 'overlaid' on other sensory information. One candidate pathway involves a light-dependent pathway in the retina. When a cryptochrome absorbs light, it generates radical pairs that affect how visual information is perceived[1]. This could give the birds something like HUD which displays magnetic field lines on top of the visual scene. Consistent with this idea, birds can only orient to magnetic fields under certain colors of light, with the color varying a bit from bird to bird[2]. It's almost as if the colored light washes out the HUD. There's another parallel pathway involving bits of magnetite in the beak[3,4]. These signals flow through the trigeminal nerve, which carries a lot of different signals; it would not be impossible for this to manifest as "pressure", as it carries touch/somatosensory information in many animals. [0] Ethologists love German and this word captures my lockdown feeling so very well. [1] https://www.sciencedirect.com/science/article/pii/S000634950076629X?via%3Dihub https://www.sciencedirect.com/science/article/pii/S000634950... [2] https://www.nature.com/articles/364525a0 https://www.nature.com/articles/364525a0 [3] https://www.sciencedirect.com/science/article/pii/S0006349507714108?via%3Dihub https://www.sciencedirect.com/science/article/pii/S000634950... [4] https://royalsocietypublishing.org/doi/10.1098/rspb.2009.0050 https://royalsocietypublishing.org/doi/10.1098/rspb.2009.005...

OmarShehata 5y ago

This is amazing. The GitHub repo has more information too, about how the common explanation of how our ears work (isolating sounds in an FFT-like-process) may be incorrect: > Contrary to what you might think, our ears don't seem to rely on an FFT-like process to extract isolated frequencies. Instead, our ears detect periodic parts in the signal, although in most cases those periodic parts closely match the FFT frequencies. There is a simple experiment that proves this point: https://github.com/soundshader/soundshader.github.io#why-acf-matches-our-perception-of-sound https://github.com/soundshader/soundshader.github.io#why-acf...

jcims 5y ago
This is my laymans test for the FFT theory. Put in earbuds or a headset and (quietly) play sounds that are slightly different in frequency (1-10hz). You will still hear a sort of ethereal beat tone between them that's different than the beat you would hear if you were listening to the same through speakers. I don't see how this would occur frequency-domain perception and it's too present (at least in my experiment) to be attributed to bone conduction across the skull.
- morelisp 5y ago
  https://en.wikipedia.org/wiki/Beat_(acoustics)#Binaural_beats https://en.wikipedia.org/wiki/Beat_(acoustics)#Binaural_beat...
  - jcims 5y ago
    Whoa! At least that gives me some comfort I wasn't imagining it. I wish I understood some of the neurological explanation of how the beats are perceived because it doesn't seem to match with my understanding of the explanation of how our ears work. If everything is pushed into the frequency domain based on the stimulation of different parts of the cochlea, where does the time domain beat emerge?
    - filoeleven 5y ago

luxuryballs 5y ago

Snek! https://soundshader.github.io/hss/gallery/bird/3.png https://soundshader.github.io/hss/gallery/bird/3.png

temporalparts 5y ago

Slight related tangent: For all those interested in birding, I highly recommend the BirdNet app available on Android and iOS! You can record and send audio data, and it will try to classify the bird for you based on the recording. In the process of recording, it will show you the bird's spectrograms, which is really cool!

algui91 5y ago
That’s really cool! trying it out now. thanks for the recommendation.

topspin 5y ago

It makes sense to me that bird song would have few harmonics. Although we may appreciate the beauty of bird song they don't sing for aesthetics. They are signaling. Concentrating energy into a narrow band delivers greater range for the same energy than harmonically rich signals. We do likewise with electromagnetic communication.

periheli0n 5y ago
When a bird sings to attract a mating partner, competing against other birds, does it not sing for aesthetics? Also, couldn’t harmonics be used to improve decoding in noisy environments? Spreading the signal over a wide band is not unheard of in man-made electromagnetic communication either.
- darig 5y ago
  Signal strength is the aesthetic. If you were a bird, you'd spend so much time trying to change what you were that you'd forget to eat your worms and die.
- karmakaze 5y ago
  When using spread-spectrum transmission it would be counterproductive to utilize harmonics. Non-sinusoidal noise at frequency F would pollute frequencies k*F. Also the transmission band is likely less than an octave.
enchiridion 5y ago
I just had two thought considering this: 1. Do we appreciate bird songs because they indicate a resource rich/hospitable environment? 2. Do birds hear the entire range of the song, or just pick up specific bands?
crazygringo 5y ago
I think the explanation is even simpler. For a loud, high-pitched sound produced from a tiny object like a bird, is it even physically possible to have the loud resonances needed for overtones and timbre? I think it's just physics. Whales are "signaling" over huge distances too but have plenty of overtones, since that's what low frequencies in huge cavities produce. I'm not sure the dichotomy between "aesthetics" and "signaling" you suggest actually exists. A peacock's plumage is certainly aesthetics but also certainly signaling. And what is a birdsong's melody if not aesthetics?

grawprog 5y ago

Those are intersting. They remind me of the sonographs we collected of bat echolocation calls when we were doing bat surveys with anabat detectors. Just by the shape of the calls you could narrow the bat down to a likely species and determine whether it was just flying around, actively hunting or about to eat a hapless insect. It was a pretty awesome device.

wombatmobile 5y ago

Um, I'd like to be able to parse this article. but I don't know what it is saying. Can anyone explain the gist of it to me in simple terms? What does this mean? "These sonograms are remarkably different from other sounds, as if birds “draw” with sound something that’s flying backwards in time."

noxToken 5y ago
Andrew Huang does a good job of explaining harmonics and overtones[0]. You only need to watch until about the 4 minute mark (from the timestamp) to get an explanation of harmonics and what the sonograms represent. The short of it is that most natural sounds product a root tone plus a varying amount of related tones above it. Our ears hear the root tone, and the other tones above it are what give the sound its uniqueness. That's why a guitar, a clarinet and 3 singers can produce the same note while sounding distinct. Birds seem to produce a natural sound without a lot of the related tones above it. Their sound is, relatively speaking, much purer than most other natural sounds. That's very unique. [0]: https://youtu.be/Wx_kugSemfY?t=95 https://youtu.be/Wx_kugSemfY?t=95
usrbin 5y ago
Audio person here. I found the post fascinating, but I wish they did more to explain what they were talking about to a layperson. Basically, all sounds that you hear are composed of many layered sine waves of different frequencies and intensities. The graphs in the post are spectrograms, which graph those frequencies over time. The Y axis represents pitch, the X axis represents time, and the brightness at any given point represents how loud that particular frequency was at that particular time. Most sounds, even seemingly simple ones, look very complex on a sonogram, like a smudged pen stroke. The images of different instruments below demonstrate this; these are all very complex sounds, even though we only hear it as a single note being played. The voice one is one of my favorites, because it shows just how weird and complicated everyday sounds can get. But bird songs are different; on a sonogram, they appear as a single line. The complexity of the bird songs here comes from the fact that they're taking a single sine wave and changing the pitch over time. Where most sounds look like a complex mix of smudged paint strokes, bird songs look like a single, precise, bouncing stroke.

klyrs 5y ago

> I don’t know if it’s a feature or a defect of the instrument. It's a feature https://monoskop.org/images/a/af/Gough_C_2007_The_violin_Chladni_patterns_plates_shells_and_sounds.pdf https://monoskop.org/images/a/af/Gough_C_2007_The_violin_Chl...

kazinator 5y ago

> However it would be a mistake to to call flute sound simple: as you see, every level has its own regular pattern that can’t be recreated with a simple mix of sinusoidal tones. The resonance of the flute's air chamber is driven by the noise of forced air stream, which is why it contains high frequencies at all. Of course you can't recreate the flute sound just with some periodic sinusoidals, because the noise component has to be present. The noise is complex and is itself filtered by the flute's chamber, in a different way depending on which valves are closed. You can play a recognizeable scale on a flute without getting a tone out of it, just using air noise, the same way you can produce a musical tune using chhhh sounds out your mouth. Those notes, or something like them, are still there in the background when a tone is being produced. There is no flute tone without them.

gbh444g 5y ago

I wanted to add that the nature of those thin horizontal layers and the patterns they form is similar to diffraction. A laser beam passing thru a thin slit forms a pattern with the sinc-wave distribution of intensity, and so is FFT of a rectangular window used in these spectrograms makes a sinc-wave shape that "diffracts" the "true" sound frequency.

birdbrain 5y ago

I don't want to detract from the images, or the work that went into this, but, er, there are a lot of types of birds. Just among passerines (songbirds), we're talking over 6,000 species. So any sweeping statement like "birds don’t seem to bother to create a complex multi-layered harmonics pattern" is practically guaranteed to be wrong. And so it is. Lots and lots of birds sing incredibly harmonically complex songs. Browse any of these (https://www.remoteenvironmentalassessmentlaboratory.com/explore/birdsong/ https://www.remoteenvironmentalassessmentlaboratory.com/expl...) if you're interested - it's a tiny sample of birds, and many, many of them do in fact have harmonically complex songs.

crazygringo 5y ago
You may be misunderstanding, though your username is very appropriate. :) "Harmonics" here isn't referring to the harmonic complexity of songs. It's referring to harmonics as in overtones, the complexity of the overtones -- the timbre. It's the voice of the bird, not the songs it sings.
- deleted 5y ago
  [deleted]
- birdbrain 5y ago
  I should have been clearer. I am also talking about moment-to-moment spectral complexity or entropy. I agree that the point TFA is suggesting seems to be about the spectral complexity at any given moment in song - what you might call the timbre - and how nearly pure the tones in their spectrograms are. (They don't specify what species they're showing. Looks like several, but I can't ID them by eye.) My point is that no, in fact, many bird species produce vocalizations that are indeed spectrally complex (beyond even just harmonic stacks) from moment-to-moment. Take a look at song from a blue jay (https://www.remoteenvironmentalassessmentlaboratory.com/explore/birdsong/bluejay.php https://www.remoteenvironmentalassessmentlaboratory.com/expl... ; not the best example, or one I produced, but an easy one to hand), particularly the syllables near the end of the clip. That's an example of a complex timbre. And lots of species produce song and calls with features like this.
5y ago

humaniania 5y ago

Are there AI projects working on mapping bird song into possible meanings to figure out a language?