5 ms·
GPT-3 trained on only English can produce French text? Citation please! I've looked for one, but I can't find it. I can, however, find more than a little inform
by rvense 5y ago
GPT-3 trained on only English can produce French text? Citation please! I've looked for one, but I can't find it. I can, however, find more than a little information suggesting that GPT-3 is trained on multilingual data.
> I think it is quite well understood that deciphering a language without external information is possible?
I don't think this is understood at all, and I would have to ask you for just one example of it. I would, in fact, say it is categorically impossible, if by "no external information" you mean that we have only text or sound.
The reason that it is not possible to extract meaning is that meaning is a substance that is not present in the text. It is more than the relations between words, in that it is connected to the world.
- rvense 5y agoI am looking more into the exact nature of the multilingual data used by GPT-3 and it's a little hard. The main paper mostly focuses on the algorithms which is less interesting to me. The key is, if you have aligned or parallel texts, then you can use structural similarities in the lexicon to bootstrap a mapping from one language to another without referring to substantive meaning (which I will maintain is how humans do translation), even if you don't have a dictionary as such. This is not surprising, but I would expect it to have severe limits.