4 ms·

Yes and theoretically if it trains on human language as well, you'd be able to translate between the two (assuming such a thing was possible, i.e assuming say d

Yes and theoretically if it trains on human language as well, you'd be able to translate between the two (assuming such a thing was possible, i.e assuming say dolphin communication was a complex language).

crazygringo 4y ago
That's the part I don't quite understand -- I know ChatGPT can translate quite well between languages. But I'm not clear on whether that's because it's been trained on a bunch of aligned translations and bilingual dictionaries and whatnot, or if it's detecting the same repeating patterns in languages and figuring out the translation mappings on its own. Is it the latter? Could a ChatGPT actually translate something like dolphin communication into semi-meaningful English, that wouldn't just be entirely nonsense?
- famouswaffles 4y ago
  It's the latter. That's why it can only be trained on 1.8% french by word count and speak fluent french. It's not that the amount of french that was in its corpus would have been enough to train it to that equivalent level alone (it wouldn't, not by a long shot). It's that there's positive transfer of knowledge. whatever it learns predicting english helps it learn other languages much faster. it's not the only display of positive transfer either. llms trained on code reason better. It's also why the open ai instruct tuning dataset was almost entirely english yet it follows instruction in other languages just fine. If there's something to be translated, a language model trained on both will translate it. it just needs to learn the concept of translation but the human side of the dataset should take care of that Really i only say theoretically because of 2 things one. first, a multimodal text-audio model that shows positive transfer between text and audio needs to be demonstrated. otherwise, this hypothetical human-dolphin model might need too much data to be feasible. 2. we're going to need a lot of dolphin speech audio.

crazygringo 4y ago

That's the part I don't quite understand -- I know ChatGPT can translate quite well between languages. But I'm not clear on whether that's because it's been trained on a bunch of aligned translations and bilingual dictionaries and whatnot, or if it's detecting the same repeating patterns in languages and figuring out the translation mappings on its own. Is it the latter? Could a ChatGPT actually translate something like dolphin communication into semi-meaningful English, that wouldn't just be entirely nonsense?

famouswaffles 4y ago
It's the latter. That's why it can only be trained on 1.8% french by word count and speak fluent french. It's not that the amount of french that was in its corpus would have been enough to train it to that equivalent level alone (it wouldn't, not by a long shot). It's that there's positive transfer of knowledge. whatever it learns predicting english helps it learn other languages much faster. it's not the only display of positive transfer either. llms trained on code reason better. It's also why the open ai instruct tuning dataset was almost entirely english yet it follows instruction in other languages just fine. If there's something to be translated, a language model trained on both will translate it. it just needs to learn the concept of translation but the human side of the dataset should take care of that Really i only say theoretically because of 2 things one. first, a multimodal text-audio model that shows positive transfer between text and audio needs to be demonstrated. otherwise, this hypothetical human-dolphin model might need too much data to be feasible. 2. we're going to need a lot of dolphin speech audio.