5 ms·

Total speculation, but I think it may be possible for transformer models to internalize a deep understanding of animal language, and possibly generate animal la

• sourcepluck 2y ago
  What makes you think this? On the face of it, what you're saying seems to be absolutely nonsensical, so I'm curious.
  • Jensson 2y ago
    Because animals doesn't use words, they use the equivalent of laughter, crying, body language, shrieks etc. So you already understand your dog and cat, as it isn't that hard to read these signals in them.
• arcastroe 2y ago
  I agree, transformer models trained solely on animal language may be able to predict the next "token" and generate coherent animal "sentences", even if we don't understand what any of those sounds/vocalizations mean. However, I do think there is a path to some sort of translation to human languages. An English dictionary may seem pointless because it defines every word using only other English words. But the meaning is contained in the _relationship_ between the words. The _relationship_ between certain animal-language tokens may look close enough to the _relationship_ between certain human-language tokens to bridge the gap, even if both languages are entirely disconnected and we've never seen a direct translation between them.
  • hun3 2y ago
    See also: word embedding and sentence embedding models
  • hackinthebochs 2y ago
    Just correlate the tokens with the state of the environment and surrounding behavior and the meaning should reveal itself. The extent that vocalizations predict subsequent environment states, or the environment predicts subsequent words, is the meaning captured by those vocalizations.
    • 082349872349872 2y ago
      to what extent do ethologists already (manually) look for these correlations?
• Sparkyte 2y ago
  Animals will probably evolve to speak well before AI figures out what animals are saying.
  • xandrius 2y ago
    Given the evolution of AI vs the evolution of animal language, the probability is actually heavily leaning towards the opposite.
    • consp 2y ago
      I would really like a _very_ detailed explanation of how AI is evolving instead of just having more computation power and shortcuts applied.
      • spacebanana7 2y ago
        Why is much detail required for the pace of change comparison? Animal language doesn’t change much. Dogs sound pretty much the same as they did 10 years ago. But AI generated sounds have changed rapidly in the past couple of years. In terms of music generation, sound to sound translation, and audio to text transcription.
        • Sparkyte 2y ago
          Actually I believe this is false. If dogs are not socialized with other dogs there is a distinct lack of language, body language varies incredibly on a per dog basis. You can say most are adopting human like mannerisms to communicate with us better. Smart glasses might be able to leverage the process of LLMS to decipher a gesture or two of dog like behavior, but it also has to learn it from the that dog's own unique vocabulary. It is different if a group of animals is socialized between themselves for a long duration and time. Why? https://en.wikipedia.org/wiki/Language_deprivation https://en.wikipedia.org/wiki/Language_deprivation You do not genetically pass down language. It has to be learned or made up.
  • victorbjorklund 2y ago
• madaxe_again 2y ago
  It’s an interesting thought - even with a translator, could their cognition and world be so alien to us as to be incompressible? I would think largely not - particularly for mammals - it’s easy to think you have nothing in common with a shrew or a whale, but we are quite recent cousins, and much of our cognitive architecture and sensorium is the same. That said, there could well be challenges and revelations beyond the obvious - for instance, can domesticated members of a species communicate with their wild cousins, or have they lost their culture/language? Cetaceans - do they talk in holograms? These are perhaps still obvious questions, but if it’s something we figure out, we may get answers to things we’ve never even considered asking.
• soulofmischief 2y ago
  Language requires a prior shared context. Communication is then optimized around relationships between those concepts. Without situational context, we might not ever determine the "meaning" for a lot of animal language. But we can still likely understand the relationships between different expressions, which may be of some use within itself to at least establish an emotional baseline for some expressions. There are already research groups who have been attempting to use embeddings and transformers to build out a vector space of specific animal languages. I think they will find some success and improve our current understanding, the question is how much context we will be able to derive. It's possible for example, that end-to-end multimodal AI would be able to train not just on animal sounds, but the body language and environmental and situational context they are in via video and other captured environmental data.