7 ms·

I can’t help but wonder if we were able to feed a lot of whale communications into LLMs with this new found layer of information if we would be able to translat

by s1mon 3y ago

I can’t help but wonder if we were able to feed a lot of whale communications into LLMs with this new found layer of information if we would be able to translate someday.

emsal 3y ago
https://youtu.be/hph9OeKjg3w https://youtu.be/hph9OeKjg3w -- here's a video about this about a research effort to accomplish precisely this.
- neodypsis 3y ago
  What do whales do if we "replay" their sounds?
  - pvaldes 3y ago
    That would be a dangerous movement. Sperm whales can kill easily an human with their sounds.
  - Loggias 3y ago
    This concept is explored in one of the episodes of Extrapolations. Fiction obviously, but interesting to think about.
mock-possum 3y ago
That’s the kind of task I really want to see LLMs used for - not just to replicate things humans can already do, but to explore things that humans haven’t figured out yet. Maybe we don’t know enough to pick up on the rhyme and reason of whale communication yet, but brute-forcing it with an AI seems like a great way to get a fresh perspective.
- cbogie 3y ago
  lol, one of my previous comments is similar here. nice to be validated as sane by one other person :)
  - el_don_almighty 3y ago
    By the power vested in me as a random comment from the internet, behold, I now declare you, "SANE!" Go forth, and insane no more.
sp332 3y ago
https://www.earthspecies.org/ https://www.earthspecies.org/ The motivating intuition was that modern machine learning can build powerful semantic representations of language which we can use to unlock communication with other species.
zer00eyz 3y ago
Well, lets get an LLM to translate one of the not understood human languages first. Could we build a chat GPT for whales to use, with this approach, sure. Does that get us any closer to translating it. NO, because without shared context to start building connections and relationships its highly unlikely that were going to find commonality.
- famouswaffles 3y ago
  >Well, lets get an LLM to translate one of not understood human languages first. You can do both. Models trained to receive and predict audio tokens alongside text are coming. You could just add such data as part of training. >Does that get us any closer to translating it. NO, because without shared context to start building connections and relationships its highly unlikely that were going to find commonality. If you trained it alongside human languages then shared context(if available) will be figured out by the model. They'll coalesce in the same shared space the way they do for human languages. Enough to translate without examples instead of just speak? Perhaps - it's not like there are examples in the training set for every lang to lang combination modern models are capable of translating.
  - zer00eyz 3y ago
    the sounds for man, and the word man can be mapped. The concept of man can be mapped from English to Chinese. Given enough of these clues, and a large enough data set it's not shocking that models get decent at inference between languages. But whales? What we are (probably) proposing is putting two languages side by side with NO context and saying "figure it out". Here is a great example of people and language and perception: https://news.mit.edu/2023/how-blue-and-green-appeared-language-1102 https://news.mit.edu/2023/how-blue-and-green-appeared-langua... Im fairly confident that unless we give an LLM a LOT of context between data sets to work with that it won't make much progress at all.
    - cbogie 3y ago
      couldn’t we approach it like (my understanding of) code breaking worked back in ww2? meta observations of recurrent symbols, like time, start/end message, etc reveal certain consistencies. then we fill in gaps through possible coherences. while ww2 code breaking contained other human contexts, perhaps like-context observations could allow for certain categories that might reveal linguistic insights of whales. like whale sounds observed consistently near boats, divers, danger, eating, play, etc. no clue about how LLM/AI systems process meta contextual training data inputs (is this even a valid phrase about AI?! haha) but i hope whale linguists have considered such an approach.
wsgeorge 3y ago
I vaguely remember reading about a research effort into this earlier this year, and I'm positive it wasn't what sp332 linked to. Would be glad if I can find the link.
halfcat 3y ago
Humans finally complete the LLM and attempt to send the first message to a whale. Humans: “We come in peace. We seek to understand and communicate with you.” No response. Humans: “We repeat. We come in peace…” WhaleGPT: “too late”
neodypsis 3y ago
Interesting experiment for Deep Mind to conduct using their multi-modal AI Gemini.
sho_hn 3y ago
Did you read the linked paper? The conclusion talks about related AI/ML research.
amelius 3y ago
At some point LLMs will become the universal translator as seen in Star Trek.
novemp 3y ago
I mean since the glorified autocomplete bots we call LLMs can't actually understand language, all we have to do is get enough samples of whale language and they can easily communicate with them.
gnramires 3y ago
It's not that simple. LLMs are great for predicting next word, but this needs a more unsupervised approach. Ultimately, you need to know the environment of whales. Sure, you know the next probable token, and even relationships between tokens, but how do they relate to what whales are doing?, in other words what does it mean?. It reminds me of the great quote by Feynman: "look at the bird"[1]. You can know the name of a bird in many different languages, but that tells you nothing about it. If you want to understand birds, you need to look at it and see what it's doing. If we want to communicate with another species, we need to understand what they're doing, why they're communicating, and from there we can begin to infer what they're communicating. It could be simply a callsign/identification/name for each individual, it could be communicating you've found a place with food, it could be communicating about predators, who knows. There could be all sorts of interesting variation and nuance to their calls. But ultimately it's necessary to look (in the general sense of having information about what it's doing -- could be through all kinds of sensors!) at the ~~bird~~ whale :) [1] Feynman tells the as a lesson from his father. (that you can know their names and still know nothing about the birds themselves). https://www.youtube.com/watch?v=ga_7j72CVlc https://www.youtube.com/watch?v=ga_7j72CVlc Bonus related video!: https://www.youtube.com/watch?v=M1TiXLGqlM4 https://www.youtube.com/watch?v=M1TiXLGqlM4

by s1mon 3y ago

I can’t help but wonder if we were able to feed a lot of whale communications into LLMs with this new found layer of information if we would be able to translate someday.

emsal 3y ago

https://youtu.be/hph9OeKjg3w https://youtu.be/hph9OeKjg3w -- here's a video about this about a research effort to accomplish precisely this.

neodypsis 3y ago
What do whales do if we "replay" their sounds?
- pvaldes 3y ago
  That would be a dangerous movement. Sperm whales can kill easily an human with their sounds.
- Loggias 3y ago
  This concept is explored in one of the episodes of Extrapolations. Fiction obviously, but interesting to think about.

mock-possum 3y ago

That’s the kind of task I really want to see LLMs used for - not just to replicate things humans can already do, but to explore things that humans haven’t figured out yet. Maybe we don’t know enough to pick up on the rhyme and reason of whale communication yet, but brute-forcing it with an AI seems like a great way to get a fresh perspective.

cbogie 3y ago
lol, one of my previous comments is similar here. nice to be validated as sane by one other person :)
- el_don_almighty 3y ago
  By the power vested in me as a random comment from the internet, behold, I now declare you, "SANE!" Go forth, and insane no more.

sp332 3y ago

https://www.earthspecies.org/ https://www.earthspecies.org/ The motivating intuition was that modern machine learning can build powerful semantic representations of language which we can use to unlock communication with other species.

zer00eyz 3y ago

Well, lets get an LLM to translate one of the not understood human languages first. Could we build a chat GPT for whales to use, with this approach, sure. Does that get us any closer to translating it. NO, because without shared context to start building connections and relationships its highly unlikely that were going to find commonality.

famouswaffles 3y ago
>Well, lets get an LLM to translate one of not understood human languages first. You can do both. Models trained to receive and predict audio tokens alongside text are coming. You could just add such data as part of training. >Does that get us any closer to translating it. NO, because without shared context to start building connections and relationships its highly unlikely that were going to find commonality. If you trained it alongside human languages then shared context(if available) will be figured out by the model. They'll coalesce in the same shared space the way they do for human languages. Enough to translate without examples instead of just speak? Perhaps - it's not like there are examples in the training set for every lang to lang combination modern models are capable of translating.
- zer00eyz 3y ago
  the sounds for man, and the word man can be mapped. The concept of man can be mapped from English to Chinese. Given enough of these clues, and a large enough data set it's not shocking that models get decent at inference between languages. But whales? What we are (probably) proposing is putting two languages side by side with NO context and saying "figure it out". Here is a great example of people and language and perception: https://news.mit.edu/2023/how-blue-and-green-appeared-language-1102 https://news.mit.edu/2023/how-blue-and-green-appeared-langua... Im fairly confident that unless we give an LLM a LOT of context between data sets to work with that it won't make much progress at all.
  - cbogie 3y ago
    couldn’t we approach it like (my understanding of) code breaking worked back in ww2? meta observations of recurrent symbols, like time, start/end message, etc reveal certain consistencies. then we fill in gaps through possible coherences. while ww2 code breaking contained other human contexts, perhaps like-context observations could allow for certain categories that might reveal linguistic insights of whales. like whale sounds observed consistently near boats, divers, danger, eating, play, etc. no clue about how LLM/AI systems process meta contextual training data inputs (is this even a valid phrase about AI?! haha) but i hope whale linguists have considered such an approach.

wsgeorge 3y ago

I vaguely remember reading about a research effort into this earlier this year, and I'm positive it wasn't what sp332 linked to. Would be glad if I can find the link.

halfcat 3y ago

Humans finally complete the LLM and attempt to send the first message to a whale. Humans: “We come in peace. We seek to understand and communicate with you.” No response. Humans: “We repeat. We come in peace…” WhaleGPT: “too late”

neodypsis 3y ago

Interesting experiment for Deep Mind to conduct using their multi-modal AI Gemini.

sho_hn 3y ago

Did you read the linked paper? The conclusion talks about related AI/ML research.

amelius 3y ago

At some point LLMs will become the universal translator as seen in Star Trek.

novemp 3y ago

I mean since the glorified autocomplete bots we call LLMs can't actually understand language, all we have to do is get enough samples of whale language and they can easily communicate with them.

gnramires 3y ago

It's not that simple. LLMs are great for predicting next word, but this needs a more unsupervised approach. Ultimately, you need to know the environment of whales. Sure, you know the next probable token, and even relationships between tokens, but how do they relate to what whales are doing?, in other words what does it mean?. It reminds me of the great quote by Feynman: "look at the bird"[1]. You can know the name of a bird in many different languages, but that tells you nothing about it. If you want to understand birds, you need to look at it and see what it's doing. If we want to communicate with another species, we need to understand what they're doing, why they're communicating, and from there we can begin to infer what they're communicating. It could be simply a callsign/identification/name for each individual, it could be communicating you've found a place with food, it could be communicating about predators, who knows. There could be all sorts of interesting variation and nuance to their calls. But ultimately it's necessary to look (in the general sense of having information about what it's doing -- could be through all kinds of sensors!) at the ~~bird~~ whale :) [1] Feynman tells the as a lesson from his father. (that you can know their names and still know nothing about the birds themselves). https://www.youtube.com/watch?v=ga_7j72CVlc https://www.youtube.com/watch?v=ga_7j72CVlc Bonus related video!: https://www.youtube.com/watch?v=M1TiXLGqlM4 https://www.youtube.com/watch?v=M1TiXLGqlM4