4 ms·

My question, why focus on "can we talk with whales" when we already have the problem of "can we talk with humans" that has a known right answer? We can have a

My question, why focus on "can we talk with whales" when we already have the problem of "can we talk with humans" that has a known right answer? We can have a spanish speaker (or speakers) sit down, feed us with hours and hours of content (heck, that's already there) and then work with AI to try and create a black box interpreter that can turn spanish to english. Has anyone done this? If the goal is universal translator why not start off with a simple case? My worry with trying to do this first with whales or other animals is we don't know the answer. Nobody can look at the whale AI translation and say "You know what, this was a good translation". Sort of the "Mars attacks" problem. All this said, I do recognize that this creates a "human bias". The way humans talk may not readily translate to animal speech (But perhaps it would for other simians?).

ethanbond 3y ago
I’m pretty sure there are dozens of startups doing AI-based translation, including just general LLMs which seem to be extremely good at this already.
- vasco 3y ago
  Those are trained differently, being exposed to translations in the training. My understanding of the person you are replying to is proposing "discovering the meaning" of Spanish by just using audio without any translations in the training. Pretending we don't know Spanish as if it were an animal/alien language.
  - ethanbond 3y ago
    Ah I see now! Thanks for the clarification.
- cogman10 3y ago
  They are solving a different problem. They are doing the "white box" translation, that is knowing good inputs and outputs. I'm talking about something more akin to what these researchers are doing. Without feeding the AI information about the correct translation, can we make an AI that can translate spanish to english? That is, what the researchers are trying to do with this whale translation.
  - ethanbond 3y ago
    I see now! Interesting idea. Thanks for clarifying :)
  - famouswaffles 3y ago
    You don't need Parallel corpora for all language pairs in a "predict the next token" LLM. What I'm saying is that if an LLM is trained on English, French and Spanish and there is Eng to French data, you don't need Eng to Spa data to get Eng to Spa translations.
    - cogman10 3y ago
      Spanish may have been a bad first language, but others like Korean, Icelandic, or even Russian would work as there is very little cross over or related languages. You'd have to be careful with the input data, though, as it would be easy to corrupt your dataset with translations if you try and do this fast.
deleted 3y ago
[deleted]
pawelwentpawel 3y ago
I wonder how much animal body language could help with that. Maybe instead of trying to focus purely on the language (like with human language translation), the algorithm could try to observe and infer the meaning from more than just audio? Dogs were domesticated long ago and can communicate with humans, sometimes purely through "facial" expressions and body movement.
- cogman10 3y ago
  I'd think it would depend a lot on the animal. Whales, for example, don't really have great eyesight. They depend a lot more on sound and have been observed communicating over long ranges (particularly because water carries sound quiet well). So it seems natural to conclude that whales would communicate more primarily through sound than other mechanisms.
  - pawelwentpawel 3y ago
    That's a great point, didn't think about that!
theptip 3y ago
But the problem is not a black box. Presumably at least some (likely most) of whale conversation is things like “I see some fish to our left” where you can measure a result, ie does the pod go left. (Some whales perform highly complex coordinated hunting strategies, which you’d think includes some verbal coordination.) Or at least, you shoot for that and maybe discover that 99% is philosophizing you can’t understand. But maybe you can bootstrap from present tense to suggest they are reminiscing about previous fish seasons. And so on.
- cogman10 3y ago
  Perhaps, though I could see there being issues like "I see fish on our left" "well, you're always wrong so we are going right" being an issue. There's even the issue with "shorthand" that might not be universalizable. For example, with a hunting strategy you could imagine "Execute the Janeway protocol" being an issue for translation. You can see some of the hunting strategy problems play out with sheep dogs. A well trained sheep herding dog can execute really impressive actions based only on specific whistles trained by the handler. [1]. It'd be wrong to conclude a specific element of language for all dogs based on the whistles. [1] https://www.youtube.com/watch?v=2YKAOMqZENo https://www.youtube.com/watch?v=2YKAOMqZENo
  - ramblenode 3y ago
    > Perhaps, though I could see there being issues like "I see fish on our left" "well, you're always wrong so we are going right" being an issue. This isn't a black box because the whalesong encodes some type of behavior that we could, in theory, observe and decode even if we didn't understand the atomic pieces of the whalesong. It's how you study any unknown language--you learn a few relations and then gradually build up a richer lexicon and grammar that gives finer-grained understanding. I'm not sure what the dog whistle example is intended to demonstrate, as even human language has a significant learned component.
  - theptip 3y ago
    I agree you could find higher-order concepts, but I don’t see any reason to assume they would be remotely as common as object-level world-descriptions. When you are asking someone to “pass the spoon, it’s to the left of the cup”, you don’t dip into a codebook and say “play 22, hut” even if you are a quarterback.
ben_w 3y ago
> Has anyone done this? Kinda. As I recall this happened by accident with French in (GPT-2? Not confident which LLM) — even though it wasn't meant to be in the training set, there were phrases just lifted directly from the other language. "Hasta la vista" et cetera have a certain je ne sais quoi, but I suggest a better Gedankenexperiment would be the language of the people of the North Sentinel Island — with whom you cannot interact by both law and them being homicidally xenophobic, and will therefore be limited to remote sensing with parabolic or laser microphones only.
- cogman10 3y ago
  Ideally, you'd use a language with native speakers who can ultimately verify the translation efforts. Perhaps a language like Icelandic, Celtic, or Korean would be better. Languages with little cross over into english while also having accessible translators. North Sentinel island would be a good stress test, but not a good first step as we can't ask them if we got the language right.

ethanbond 3y ago

I’m pretty sure there are dozens of startups doing AI-based translation, including just general LLMs which seem to be extremely good at this already.

vasco 3y ago
Those are trained differently, being exposed to translations in the training. My understanding of the person you are replying to is proposing "discovering the meaning" of Spanish by just using audio without any translations in the training. Pretending we don't know Spanish as if it were an animal/alien language.
- ethanbond 3y ago
  Ah I see now! Thanks for the clarification.
cogman10 3y ago
They are solving a different problem. They are doing the "white box" translation, that is knowing good inputs and outputs. I'm talking about something more akin to what these researchers are doing. Without feeding the AI information about the correct translation, can we make an AI that can translate spanish to english? That is, what the researchers are trying to do with this whale translation.
- ethanbond 3y ago
  I see now! Interesting idea. Thanks for clarifying :)
- famouswaffles 3y ago
  You don't need Parallel corpora for all language pairs in a "predict the next token" LLM. What I'm saying is that if an LLM is trained on English, French and Spanish and there is Eng to French data, you don't need Eng to Spa data to get Eng to Spa translations.
  - cogman10 3y ago
    Spanish may have been a bad first language, but others like Korean, Icelandic, or even Russian would work as there is very little cross over or related languages. You'd have to be careful with the input data, though, as it would be easy to corrupt your dataset with translations if you try and do this fast.

deleted 3y ago

[deleted]

pawelwentpawel 3y ago

I wonder how much animal body language could help with that. Maybe instead of trying to focus purely on the language (like with human language translation), the algorithm could try to observe and infer the meaning from more than just audio? Dogs were domesticated long ago and can communicate with humans, sometimes purely through "facial" expressions and body movement.

cogman10 3y ago
I'd think it would depend a lot on the animal. Whales, for example, don't really have great eyesight. They depend a lot more on sound and have been observed communicating over long ranges (particularly because water carries sound quiet well). So it seems natural to conclude that whales would communicate more primarily through sound than other mechanisms.
- pawelwentpawel 3y ago
  That's a great point, didn't think about that!

theptip 3y ago

But the problem is not a black box. Presumably at least some (likely most) of whale conversation is things like “I see some fish to our left” where you can measure a result, ie does the pod go left. (Some whales perform highly complex coordinated hunting strategies, which you’d think includes some verbal coordination.) Or at least, you shoot for that and maybe discover that 99% is philosophizing you can’t understand. But maybe you can bootstrap from present tense to suggest they are reminiscing about previous fish seasons. And so on.

cogman10 3y ago
Perhaps, though I could see there being issues like "I see fish on our left" "well, you're always wrong so we are going right" being an issue. There's even the issue with "shorthand" that might not be universalizable. For example, with a hunting strategy you could imagine "Execute the Janeway protocol" being an issue for translation. You can see some of the hunting strategy problems play out with sheep dogs. A well trained sheep herding dog can execute really impressive actions based only on specific whistles trained by the handler. [1]. It'd be wrong to conclude a specific element of language for all dogs based on the whistles. [1] https://www.youtube.com/watch?v=2YKAOMqZENo https://www.youtube.com/watch?v=2YKAOMqZENo
- ramblenode 3y ago
  > Perhaps, though I could see there being issues like "I see fish on our left" "well, you're always wrong so we are going right" being an issue. This isn't a black box because the whalesong encodes some type of behavior that we could, in theory, observe and decode even if we didn't understand the atomic pieces of the whalesong. It's how you study any unknown language--you learn a few relations and then gradually build up a richer lexicon and grammar that gives finer-grained understanding. I'm not sure what the dog whistle example is intended to demonstrate, as even human language has a significant learned component.
- theptip 3y ago
  I agree you could find higher-order concepts, but I don’t see any reason to assume they would be remotely as common as object-level world-descriptions. When you are asking someone to “pass the spoon, it’s to the left of the cup”, you don’t dip into a codebook and say “play 22, hut” even if you are a quarterback.

ben_w 3y ago

> Has anyone done this? Kinda. As I recall this happened by accident with French in (GPT-2? Not confident which LLM) — even though it wasn't meant to be in the training set, there were phrases just lifted directly from the other language. "Hasta la vista" et cetera have a certain je ne sais quoi, but I suggest a better Gedankenexperiment would be the language of the people of the North Sentinel Island — with whom you cannot interact by both law and them being homicidally xenophobic, and will therefore be limited to remote sensing with parabolic or laser microphones only.

cogman10 3y ago
Ideally, you'd use a language with native speakers who can ultimately verify the translation efforts. Perhaps a language like Icelandic, Celtic, or Korean would be better. Languages with little cross over into english while also having accessible translators. North Sentinel island would be a good stress test, but not a good first step as we can't ask them if we got the language right.