3 ms·

Well, lets get an LLM to translate one of the not understood human languages first. Could we build a chat GPT for whales to use, with this approach, sure. Does

Well, lets get an LLM to translate one of the not understood human languages first. Could we build a chat GPT for whales to use, with this approach, sure. Does that get us any closer to translating it. NO, because without shared context to start building connections and relationships its highly unlikely that were going to find commonality.

famouswaffles 3y ago
>Well, lets get an LLM to translate one of not understood human languages first. You can do both. Models trained to receive and predict audio tokens alongside text are coming. You could just add such data as part of training. >Does that get us any closer to translating it. NO, because without shared context to start building connections and relationships its highly unlikely that were going to find commonality. If you trained it alongside human languages then shared context(if available) will be figured out by the model. They'll coalesce in the same shared space the way they do for human languages. Enough to translate without examples instead of just speak? Perhaps - it's not like there are examples in the training set for every lang to lang combination modern models are capable of translating.
- zer00eyz 3y ago
  the sounds for man, and the word man can be mapped. The concept of man can be mapped from English to Chinese. Given enough of these clues, and a large enough data set it's not shocking that models get decent at inference between languages. But whales? What we are (probably) proposing is putting two languages side by side with NO context and saying "figure it out". Here is a great example of people and language and perception: https://news.mit.edu/2023/how-blue-and-green-appeared-language-1102 https://news.mit.edu/2023/how-blue-and-green-appeared-langua... Im fairly confident that unless we give an LLM a LOT of context between data sets to work with that it won't make much progress at all.
  - cbogie 3y ago
    couldn’t we approach it like (my understanding of) code breaking worked back in ww2? meta observations of recurrent symbols, like time, start/end message, etc reveal certain consistencies. then we fill in gaps through possible coherences. while ww2 code breaking contained other human contexts, perhaps like-context observations could allow for certain categories that might reveal linguistic insights of whales. like whale sounds observed consistently near boats, divers, danger, eating, play, etc. no clue about how LLM/AI systems process meta contextual training data inputs (is this even a valid phrase about AI?! haha) but i hope whale linguists have considered such an approach.
  - famouswaffles 3y ago
    >the sounds for man, and the word man can be mapped. The concept of man can be mapped from English to Chinese. Given enough of these clues, and a large enough data set it's not shocking that models get decent at inference between languages. 1. Many such concepts cannot be directly mapped between different languages especially with distant language pairs. 2. Language Models and Image models even if both are only trained on their respective modality learn structural representations so similar, you can connect them with a simple linear layer. That's it. Entirely different modalities https://arxiv.org/abs/2209.15162 https://arxiv.org/abs/2209.15162 https://arxiv.org/abs/2304.08485 https://arxiv.org/abs/2304.08485 I think you are severely underestimating the extent to which representations can group in neural networks. They're mammals, they're social animals, they see. You're assuming a level of alienness we don't have the knowledge or understanding to truly ascertain.
acadapter 3y ago
Not understood human languages are in general languages with a low amount of source material.
- cbogie 3y ago
  given unrestricted compute, could an AI attempt to decipher a completely unknown language through interpreting O(n) brute iterations and reporting versions that are coherent to human heuristics?
  - water-your-self 3y ago
    Its a lack of information to verify correctness not a lack of attempts. One plank of wood is not enough to determine solely if it was intended to be part of a house or part of a boat
  - adastra22 3y ago
    There are uncountably many “valid” interpretations by this approach. With scant source material there is no way to verify the guessed translation as correct or not. It’s like “guessing” the encryption key of a one-time pad.
Teever 3y ago
I was under the impression that the reason why we don't understand the languages that we don't understand is because we lack sufficient records of them.
elcomet 3y ago
We can do this already : https://arxiv.org/abs/1711.00043 https://arxiv.org/abs/1711.00043
- zer00eyz 3y ago
  The technique in that paper, is likely not generalizable. It works for EN/FR, less well for EN/DE... because English has a huge overlap with these two (more French than German) for its word by word translation. With something like Chinese, and enough time it might be able to guess its way in due to overlaps in the training data. If you took two completely random corpus's from two languages, and jammed them into this model and handed that system to someone who only knew one language, the approach would fall flat on its face.
currymj 3y ago
a major focus of the project is to also monitor the whales' physical movement, in order to put the particular sounds in the context of actions.

famouswaffles 3y ago

>Well, lets get an LLM to translate one of not understood human languages first. You can do both. Models trained to receive and predict audio tokens alongside text are coming. You could just add such data as part of training. >Does that get us any closer to translating it. NO, because without shared context to start building connections and relationships its highly unlikely that were going to find commonality. If you trained it alongside human languages then shared context(if available) will be figured out by the model. They'll coalesce in the same shared space the way they do for human languages. Enough to translate without examples instead of just speak? Perhaps - it's not like there are examples in the training set for every lang to lang combination modern models are capable of translating.

zer00eyz 3y ago
the sounds for man, and the word man can be mapped. The concept of man can be mapped from English to Chinese. Given enough of these clues, and a large enough data set it's not shocking that models get decent at inference between languages. But whales? What we are (probably) proposing is putting two languages side by side with NO context and saying "figure it out". Here is a great example of people and language and perception: https://news.mit.edu/2023/how-blue-and-green-appeared-language-1102 https://news.mit.edu/2023/how-blue-and-green-appeared-langua... Im fairly confident that unless we give an LLM a LOT of context between data sets to work with that it won't make much progress at all.
- cbogie 3y ago
  couldn’t we approach it like (my understanding of) code breaking worked back in ww2? meta observations of recurrent symbols, like time, start/end message, etc reveal certain consistencies. then we fill in gaps through possible coherences. while ww2 code breaking contained other human contexts, perhaps like-context observations could allow for certain categories that might reveal linguistic insights of whales. like whale sounds observed consistently near boats, divers, danger, eating, play, etc. no clue about how LLM/AI systems process meta contextual training data inputs (is this even a valid phrase about AI?! haha) but i hope whale linguists have considered such an approach.
- famouswaffles 3y ago
  >the sounds for man, and the word man can be mapped. The concept of man can be mapped from English to Chinese. Given enough of these clues, and a large enough data set it's not shocking that models get decent at inference between languages. 1. Many such concepts cannot be directly mapped between different languages especially with distant language pairs. 2. Language Models and Image models even if both are only trained on their respective modality learn structural representations so similar, you can connect them with a simple linear layer. That's it. Entirely different modalities https://arxiv.org/abs/2209.15162 https://arxiv.org/abs/2209.15162 https://arxiv.org/abs/2304.08485 https://arxiv.org/abs/2304.08485 I think you are severely underestimating the extent to which representations can group in neural networks. They're mammals, they're social animals, they see. You're assuming a level of alienness we don't have the knowledge or understanding to truly ascertain.

acadapter 3y ago

Not understood human languages are in general languages with a low amount of source material.

cbogie 3y ago
given unrestricted compute, could an AI attempt to decipher a completely unknown language through interpreting O(n) brute iterations and reporting versions that are coherent to human heuristics?
- water-your-self 3y ago
  Its a lack of information to verify correctness not a lack of attempts. One plank of wood is not enough to determine solely if it was intended to be part of a house or part of a boat
- adastra22 3y ago
  There are uncountably many “valid” interpretations by this approach. With scant source material there is no way to verify the guessed translation as correct or not. It’s like “guessing” the encryption key of a one-time pad.

Teever 3y ago

I was under the impression that the reason why we don't understand the languages that we don't understand is because we lack sufficient records of them.

elcomet 3y ago

We can do this already : https://arxiv.org/abs/1711.00043 https://arxiv.org/abs/1711.00043

zer00eyz 3y ago
The technique in that paper, is likely not generalizable. It works for EN/FR, less well for EN/DE... because English has a huge overlap with these two (more French than German) for its word by word translation. With something like Chinese, and enough time it might be able to guess its way in due to overlaps in the training data. If you took two completely random corpus's from two languages, and jammed them into this model and handed that system to someone who only knew one language, the approach would fall flat on its face.

currymj 3y ago

a major focus of the project is to also monitor the whales' physical movement, in order to put the particular sounds in the context of actions.