4 ms·

I think OP was imagining using a LLM trained only on animal sound data, not using a LLM trained on human data to interpret animal data.

qayxc 4y ago
Which begs the question what the added benefit would be from such an approach, i.e. what could we even learn from that compared to other methods? To quote from an article in The Royal Society: > [...] bird song evolution can also be understood as a response to natural selection, as when oscine passerine species in urban habitats raise the frequency of their song as a means to overcome traffic noise. Things like changes in the vocalisation can just be adjustments to the environment or simply due to sexual selection. How would an LLM be able to capture this, given that it would lack such additional information? Then there's this quote from the same article: > In contrast to the expectation of gradual change through cultural or genetic drift over time, the results instead demonstrate that plastic traits such as song can exhibit punctuated evolution, with bursts of trait divergence interrupting extended periods of stasis. So even if a model was trained on (regional) bird dialects, the data could become obsolete basically over night, as the structure and patterns might change dramatically between breeding seasons. The researchers used statistical modelling (classical Gaussian mixture models) to analyse the data. I'm a bit lost as to how LLMs could be applied in this context (in that I have no idea what kind of scientific questions they could answer). The full study can be found here: https://royalsocietypublishing.org/doi/10.1098/rspb.2021.2062 https://royalsocietypublishing.org/doi/10.1098/rspb.2021.206...
- nerdponx 4y ago
  It's not unreasonable to expect that a big enough model could learn some kind of useful feature space for bird vocalizations. You are essentially replacing a parsimonious quasi-parametric model (Gaussian process) with an enormous fully nonparametric one. Transformer units should be as well-suited for this as for any other sequence learning task.
goatlover 4y ago
But the LLM would need to translate the sounds to human language, and that's where the issue would arise.
- jerf 4y ago
  Yes, this was my logic. A model trained on bird song may make birds turn their heads when it babbles something in their frame of reference, but if you want to "use LLM" to make it human comprehensible you need a human LLM at some point, which will impose humanity on its input regardless of what you feed it, because all a human LLM can represent is human language. A human LLM can't help but make whatever it is used on human. A bird LLM would have the same effect; no matter what you feed it, you'll get something in the bird input's space.
  - famouswaffles 4y ago
    Not sure you're really getting the idea. In this scenario, there wouldn't be human llm and dolphin llm, you would train human speech and dolphin communication in one model. sound tokens are sound tokens. pre-trained transformers don't discriminate.
    - goatlover 4y ago
      The concern would be that it would gnerate a bunch of spurious correlations between the two. You would be mixing two kinds of datasets, where it’s not clear that the dolphin sounds are linguistic. It’s also not clear that LLMs would correctly map human language to a non-human one. Depends on your theory of meaning. Wittgenstein would say if a lion could talk, we would not understand it, because lmeaning is use, and lions would use language differently than humans.

qayxc 4y ago

Which begs the question what the added benefit would be from such an approach, i.e. what could we even learn from that compared to other methods? To quote from an article in The Royal Society: > [...] bird song evolution can also be understood as a response to natural selection, as when oscine passerine species in urban habitats raise the frequency of their song as a means to overcome traffic noise. Things like changes in the vocalisation can just be adjustments to the environment or simply due to sexual selection. How would an LLM be able to capture this, given that it would lack such additional information? Then there's this quote from the same article: > In contrast to the expectation of gradual change through cultural or genetic drift over time, the results instead demonstrate that plastic traits such as song can exhibit punctuated evolution, with bursts of trait divergence interrupting extended periods of stasis. So even if a model was trained on (regional) bird dialects, the data could become obsolete basically over night, as the structure and patterns might change dramatically between breeding seasons. The researchers used statistical modelling (classical Gaussian mixture models) to analyse the data. I'm a bit lost as to how LLMs could be applied in this context (in that I have no idea what kind of scientific questions they could answer). The full study can be found here: https://royalsocietypublishing.org/doi/10.1098/rspb.2021.2062 https://royalsocietypublishing.org/doi/10.1098/rspb.2021.206...

nerdponx 4y ago
It's not unreasonable to expect that a big enough model could learn some kind of useful feature space for bird vocalizations. You are essentially replacing a parsimonious quasi-parametric model (Gaussian process) with an enormous fully nonparametric one. Transformer units should be as well-suited for this as for any other sequence learning task.

goatlover 4y ago

But the LLM would need to translate the sounds to human language, and that's where the issue would arise.

jerf 4y ago
Yes, this was my logic. A model trained on bird song may make birds turn their heads when it babbles something in their frame of reference, but if you want to "use LLM" to make it human comprehensible you need a human LLM at some point, which will impose humanity on its input regardless of what you feed it, because all a human LLM can represent is human language. A human LLM can't help but make whatever it is used on human. A bird LLM would have the same effect; no matter what you feed it, you'll get something in the bird input's space.
- famouswaffles 4y ago
  Not sure you're really getting the idea. In this scenario, there wouldn't be human llm and dolphin llm, you would train human speech and dolphin communication in one model. sound tokens are sound tokens. pre-trained transformers don't discriminate.
  - goatlover 4y ago
    The concern would be that it would gnerate a bunch of spurious correlations between the two. You would be mixing two kinds of datasets, where it’s not clear that the dolphin sounds are linguistic. It’s also not clear that LLMs would correctly map human language to a non-human one. Depends on your theory of meaning. Wittgenstein would say if a lion could talk, we would not understand it, because lmeaning is use, and lions would use language differently than humans.