I am looking more into the exact nature of the multilingual data used by GPT-3 and it's a little hard. The main paper mostly focuses on the algorithms which is

I am looking more into the exact nature of the multilingual data used by GPT-3 and it's a little hard. The main paper mostly focuses on the algorithms which is less interesting to me. The key is, if you have aligned or parallel texts, then you can use structural similarities in the lexicon to bootstrap a mapping from one language to another without referring to substantive meaning (which I will maintain is how humans do translation), even if you don't have a dictionary as such. This is not surprising, but I would expect it to have severe limits.