6 ms·

Transcribing bird sound as human speech

70 points by tintinnabula 4y ago

jcims 4y ago
Related (the inverse actually) https://www.windytan.com/2021/03/speech-to-birdsong-conversion.html https://www.windytan.com/2021/03/speech-to-birdsong-conversi...
- PebblesRox 4y ago
  I was expecting this link: https://en.m.wikipedia.org/wiki/Whistled_language https://en.m.wikipedia.org/wiki/Whistled_language
VyseofArcadia 4y ago
Dig, if you will, the picture Of calls written down in ink.
- maxbond 4y ago
  Transcribed by a token predictor, A machine that pretends to think.
tgv 4y ago
A discussion of transcribing bird song is not complete without a reference to Messiæn's Catalogue d'Oiseaux (bird catalog), for which he transcribed bird song to pitch (https://www.oliviermessiaen.org/birdsong https://www.oliviermessiaen.org/birdsong), and used those as starting point for composition (e.g. https://www.youtube.com/watch?v=UmrAL2WNjGs https://www.youtube.com/watch?v=UmrAL2WNjGs). For the people who expect gentle bird imitation: it's much wilder than that.
- dhosek 4y ago
  Messiaen in general is wild, but his use of birdsong is especially cool. Long before I knew about this, I had an idea of writing a cello/flute duet based on birdsong, but at the time, it was hard finding good recordings let alone transcriptions of birdsong. I do think that musical notation might be a good starting point for birdsong transcription though.
  - nerdponx 4y ago
    This output to me sounds a lot like a contemporary "lots of complicated dramatic notes"-style of composer imposing that specific style on snippets that loosely resemble bird calls. It doesn't sound like birds at all. Maybe I'm just not smart enough for this stuff. Or maybe the composer has a particularly dramatic imagination and is using the birds just as a starting point.
    - convolvatron 4y ago
      I don't think its a question of intelligence. you have to spend some time listening to messaien and late romantic/contemporary music in general before you start to 'learn the language'. I think its worth the minor effort if you think there is nothing new under the sun, but it all your choice :)
    - tgv 4y ago
      To get into Messiaen, try his Turangalîla symphony. It's more accessible, and --if you're so inclined-- almost ecstatic. Then the Quator pour le fin du temps, which was written under particularly difficult conditions. His more religiously oriented work is also more accessible. Well, when I write "more accessible" that indeed means the rest is harder to get into. No shame, though. It's a bit of an acquired taste. There are many 20th century composers (especially the serialists) I cannot stand. I can't hear the music in it. But Messiaen did get to me. Perhaps it's his "colours" or the Debussy-like chords.
      - orbisvicis 4y ago
        I think the dynamic range of multiple instruments naturally invites "lots of complicated dramatic notes", which you can find even in romantic music. Of Messiaen's works, my favorite remains "Vingt regards sur l'Enfant-Jésus". Of course the symphony also introduces many new and alluring timbres not found in the piano. In "Catalogue d'Oiseaux" I myself also struggled to find more than a passing resemblance to bird song (i.e. beyond more familiar song found in the treble of l'Alouette Lulu), so thanks for those links. As for ecstatic music, I wonder how you'd appreciate the music of Maurice Ohana, specifically "Messe", which despite being an operatic (or choral?) piece, I find calming. I believe John Sichel's "Oiseaux Ordinaires" is homage to Messiaen's "Oiseaux Exotiques", though once again I'm not sure how much actual birdsong it incorporates. I've always wondered how much of music's aesthetic is inherent, and how much relates to extrinsic factors. Normally you hear a piece and if you like it, you think, "hmm, that's good music". But maybe you only liked it because you're used to that type of music. So if you want to listen to music from a new genre perhaps you listen to it a few times, letting an appreciation percolate from whatever place we draw our musical inspiration. Then, because a lot of people say "this is a good genre; good music" and the genre is large with some shit pieces but also masterpieces, you come to believe that the genre has value. But some genres very few people come to appreciate, and that includes serialism, and I wonder... Is serialism lacking an inherent aesthetic, or is it just too different to easily appreciate? For example I played, and came to love, Schoenberg's "Six Little Pieces for Piano" (op 19). My interpretation was idiosyncratic; like no one else's. But looking on YouTube I can't find any performances that I enjoy or would recommend. So does my appreciation for this music reflect the blood and sweat that I've put into it, or does it reflect the beauty and understanding of the underlying musical theory? The question is: does serialism need to be played to be appreciated?
crazygringo 4y ago
Tangential, but -- it occurs to me, is there any way to apply the methods behind ChatGPT to animal communication, and is there any way we could learn anything about meaning from that? It seems like AI ought to be able to "autocomplete" birdsong or whale sounds just as well as anything else, given enough training data. Could that then be used to help discover underlying grammars, etc.?
- lamename 4y ago
  You might be interested in work being done by the Earth Species Project https://www.earthspecies.org/what-we-do/projects https://www.earthspecies.org/what-we-do/projects
  - crazygringo 4y ago
    That is fascinating, thank you! Self-supervised learning that could correlate animal sounds with behavior. Exactly the kind of thing I was thinking about.
- qayxc 4y ago
  Careful with anthropomorphising. Communication is much more than vocalisation of abstract thought (i.e. the way humans mostly use it). Animals don't communicate the same way we do and it might be very misleading to associate concepts such as grammar with their utterances. While some species use distinct sounds to express certain situations (like "careful, there's an airborne predator" vs "careful, there's a snake"), not all "verbal" expressions serve such purpose. The general consensus in contemporary science is that bird songs in particular are utilities to attract mates, mark territory, and the like; not exchanging information (other than things like "this is my nesting place" or "I'm healthy and available", but not in a sense of words or sentences, but more like the acoustic equivalent of gesturing).
  - jerf 4y ago
    "Careful with anthropomorphising." The confabulation elements of this particular AI technology would be particularly concerning in this use case. This would be the most literal anthropomorphization you could imagine. Applying a strictly human language model to a non-human will inevitably result in humanization of the target by the nature of the bias of the system. I fed "You are a rock. What might you like to communicate to human beings?" to ChatGPT and it answered just fine, off of nothing more than that input, giving one of its typically highly sanctimonious and high-handed lectures about environmentalism and managing to work in complaints about climate change, as naturally a rock would deeply care about. (It has now even titled my session "Rock's Wisdom To Humans". Good lord, ChatGPT has certainly become an annoying prick.) This has everything to do with the biases in the technology and nothing to do with the thing putatively doing the communication. Feeding it some sort of data from a real whale isn't going to do any better; it'll happily confabulate away. "Bias" in this case has a more technical meaning, which refers to the representational capability of the system, not political biases intrinsically. It so happen that ChatGPT immediately leaps to political topics, which is doing a great job of obscuring the way I'm trying to use the term, but that's because of its own internal structures which have been tuned in that direction. This, in fact, accidentally serves as as good an example as anything of how highly tuned ChatGPT has become to give politically correct answers, given the political neutrality of the prompt. But what I'm referring to here is just that a human language model is going to be biased in human ways. It can't help but turn anything you feed it into a human perspective, so using it for trying to understand non-human perspectives and communication is going to be intrinsically a mistake.
leftbit 4y ago
Reminded me of this interesting bird song visualization project: https://github.com/soundshader/soundshader.github.io/tree/master/birdglyph https://github.com/soundshader/soundshader.github.io/tree/ma... Less information than a spectrogram, but really visually appealing. :)
- mito88 4y ago
  this came to mind. https://music.youtube.com/playlist?list=OLAK5uy_lGpNxYqh-Ho5Hs1hej-YoGI-usA-pq9KY&feature=share https://music.youtube.com/playlist?list=OLAK5uy_lGpNxYqh-Ho5...
- djtriptych 4y ago
  wow this is so cool. thanks for sharing
sroussey 4y ago
Who is using machine learning on bird sounds/songs? I have three parrots and understood their happy sounds, angry sounds, and love you sounds. But missing so much more…
- blacksmith_tb 4y ago
  Amen to that, I absolutely want a "tell me what those crows are generally talking about" app.
emmelaich 4y ago
For use of birds sounds by people in music ... I absolutely love this: https://www.youtube.com/watch?v=Hxg1dL_x0gw https://www.youtube.com/watch?v=Hxg1dL_x0gw AkhaBraka - Ukrainian Folk

70 points by tintinnabula 4y ago

Related (the inverse actually) https://www.windytan.com/2021/03/speech-to-birdsong-conversion.html https://www.windytan.com/2021/03/speech-to-birdsong-conversi...

PebblesRox 4y ago
I was expecting this link: https://en.m.wikipedia.org/wiki/Whistled_language https://en.m.wikipedia.org/wiki/Whistled_language

VyseofArcadia 4y ago

Dig, if you will, the picture Of calls written down in ink.

maxbond 4y ago
Transcribed by a token predictor, A machine that pretends to think.

tgv 4y ago

A discussion of transcribing bird song is not complete without a reference to Messiæn's Catalogue d'Oiseaux (bird catalog), for which he transcribed bird song to pitch (https://www.oliviermessiaen.org/birdsong https://www.oliviermessiaen.org/birdsong), and used those as starting point for composition (e.g. https://www.youtube.com/watch?v=UmrAL2WNjGs https://www.youtube.com/watch?v=UmrAL2WNjGs). For the people who expect gentle bird imitation: it's much wilder than that.

dhosek 4y ago
Messiaen in general is wild, but his use of birdsong is especially cool. Long before I knew about this, I had an idea of writing a cello/flute duet based on birdsong, but at the time, it was hard finding good recordings let alone transcriptions of birdsong. I do think that musical notation might be a good starting point for birdsong transcription though.
- nerdponx 4y ago
  This output to me sounds a lot like a contemporary "lots of complicated dramatic notes"-style of composer imposing that specific style on snippets that loosely resemble bird calls. It doesn't sound like birds at all. Maybe I'm just not smart enough for this stuff. Or maybe the composer has a particularly dramatic imagination and is using the birds just as a starting point.
  - convolvatron 4y ago
    I don't think its a question of intelligence. you have to spend some time listening to messaien and late romantic/contemporary music in general before you start to 'learn the language'. I think its worth the minor effort if you think there is nothing new under the sun, but it all your choice :)
  - tgv 4y ago
    To get into Messiaen, try his Turangalîla symphony. It's more accessible, and --if you're so inclined-- almost ecstatic. Then the Quator pour le fin du temps, which was written under particularly difficult conditions. His more religiously oriented work is also more accessible. Well, when I write "more accessible" that indeed means the rest is harder to get into. No shame, though. It's a bit of an acquired taste. There are many 20th century composers (especially the serialists) I cannot stand. I can't hear the music in it. But Messiaen did get to me. Perhaps it's his "colours" or the Debussy-like chords.
    - orbisvicis 4y ago
      I think the dynamic range of multiple instruments naturally invites "lots of complicated dramatic notes", which you can find even in romantic music. Of Messiaen's works, my favorite remains "Vingt regards sur l'Enfant-Jésus". Of course the symphony also introduces many new and alluring timbres not found in the piano. In "Catalogue d'Oiseaux" I myself also struggled to find more than a passing resemblance to bird song (i.e. beyond more familiar song found in the treble of l'Alouette Lulu), so thanks for those links. As for ecstatic music, I wonder how you'd appreciate the music of Maurice Ohana, specifically "Messe", which despite being an operatic (or choral?) piece, I find calming. I believe John Sichel's "Oiseaux Ordinaires" is homage to Messiaen's "Oiseaux Exotiques", though once again I'm not sure how much actual birdsong it incorporates. I've always wondered how much of music's aesthetic is inherent, and how much relates to extrinsic factors. Normally you hear a piece and if you like it, you think, "hmm, that's good music". But maybe you only liked it because you're used to that type of music. So if you want to listen to music from a new genre perhaps you listen to it a few times, letting an appreciation percolate from whatever place we draw our musical inspiration. Then, because a lot of people say "this is a good genre; good music" and the genre is large with some shit pieces but also masterpieces, you come to believe that the genre has value. But some genres very few people come to appreciate, and that includes serialism, and I wonder... Is serialism lacking an inherent aesthetic, or is it just too different to easily appreciate? For example I played, and came to love, Schoenberg's "Six Little Pieces for Piano" (op 19). My interpretation was idiosyncratic; like no one else's. But looking on YouTube I can't find any performances that I enjoy or would recommend. So does my appreciation for this music reflect the blood and sweat that I've put into it, or does it reflect the beauty and understanding of the underlying musical theory? The question is: does serialism need to be played to be appreciated?

crazygringo 4y ago

Tangential, but -- it occurs to me, is there any way to apply the methods behind ChatGPT to animal communication, and is there any way we could learn anything about meaning from that? It seems like AI ought to be able to "autocomplete" birdsong or whale sounds just as well as anything else, given enough training data. Could that then be used to help discover underlying grammars, etc.?

lamename 4y ago
You might be interested in work being done by the Earth Species Project https://www.earthspecies.org/what-we-do/projects https://www.earthspecies.org/what-we-do/projects
- crazygringo 4y ago
  That is fascinating, thank you! Self-supervised learning that could correlate animal sounds with behavior. Exactly the kind of thing I was thinking about.
qayxc 4y ago
Careful with anthropomorphising. Communication is much more than vocalisation of abstract thought (i.e. the way humans mostly use it). Animals don't communicate the same way we do and it might be very misleading to associate concepts such as grammar with their utterances. While some species use distinct sounds to express certain situations (like "careful, there's an airborne predator" vs "careful, there's a snake"), not all "verbal" expressions serve such purpose. The general consensus in contemporary science is that bird songs in particular are utilities to attract mates, mark territory, and the like; not exchanging information (other than things like "this is my nesting place" or "I'm healthy and available", but not in a sense of words or sentences, but more like the acoustic equivalent of gesturing).
- jerf 4y ago
  "Careful with anthropomorphising." The confabulation elements of this particular AI technology would be particularly concerning in this use case. This would be the most literal anthropomorphization you could imagine. Applying a strictly human language model to a non-human will inevitably result in humanization of the target by the nature of the bias of the system. I fed "You are a rock. What might you like to communicate to human beings?" to ChatGPT and it answered just fine, off of nothing more than that input, giving one of its typically highly sanctimonious and high-handed lectures about environmentalism and managing to work in complaints about climate change, as naturally a rock would deeply care about. (It has now even titled my session "Rock's Wisdom To Humans". Good lord, ChatGPT has certainly become an annoying prick.) This has everything to do with the biases in the technology and nothing to do with the thing putatively doing the communication. Feeding it some sort of data from a real whale isn't going to do any better; it'll happily confabulate away. "Bias" in this case has a more technical meaning, which refers to the representational capability of the system, not political biases intrinsically. It so happen that ChatGPT immediately leaps to political topics, which is doing a great job of obscuring the way I'm trying to use the term, but that's because of its own internal structures which have been tuned in that direction. This, in fact, accidentally serves as as good an example as anything of how highly tuned ChatGPT has become to give politically correct answers, given the political neutrality of the prompt. But what I'm referring to here is just that a human language model is going to be biased in human ways. It can't help but turn anything you feed it into a human perspective, so using it for trying to understand non-human perspectives and communication is going to be intrinsically a mistake.

leftbit 4y ago

Reminded me of this interesting bird song visualization project: https://github.com/soundshader/soundshader.github.io/tree/master/birdglyph https://github.com/soundshader/soundshader.github.io/tree/ma... Less information than a spectrogram, but really visually appealing. :)

mito88 4y ago
this came to mind. https://music.youtube.com/playlist?list=OLAK5uy_lGpNxYqh-Ho5Hs1hej-YoGI-usA-pq9KY&feature=share https://music.youtube.com/playlist?list=OLAK5uy_lGpNxYqh-Ho5...
djtriptych 4y ago
wow this is so cool. thanks for sharing

sroussey 4y ago

Who is using machine learning on bird sounds/songs? I have three parrots and understood their happy sounds, angry sounds, and love you sounds. But missing so much more…

blacksmith_tb 4y ago
Amen to that, I absolutely want a "tell me what those crows are generally talking about" app.

emmelaich 4y ago

For use of birds sounds by people in music ... I absolutely love this: https://www.youtube.com/watch?v=Hxg1dL_x0gw https://www.youtube.com/watch?v=Hxg1dL_x0gw AkhaBraka - Ukrainian Folk