I really think you need to research a bit more about how statistical language models work. The reason this models are able to learn other languages so quickly

I really think you need to research a bit more about how statistical language models work. The reason this models are able to learn other languages so quickly is that they're using the majority language to do much of the heavy lifting in defining the overall latent language space. Once this is established, adding new languages requires much less information since you only have to learn to encode and slightly tweak the latent presentation you learn. Again the view of statistical NLP, from LSA to GPT-N is that the words we write are generated from transformation of a latent space. For LSA this transformation is linear and the space is low dimensional, and the model only learn to encode words to the space. For GPT-N this transformation is non-linear, the latent space is very high dimensional and the model learns both the encoder and a decoder allowing it to generate language from the latent space. If you have enough language to model this latent space well, then learning additional languages requires very little training data since the space itself is well modeled, you just need to learn the details of the encoder/decoder. That is you've learned the abstract point in high dimensional space that represents the English word "cat", you just need to learn that "猫" points to the same place. Of course these models aren't just representing words/tokens without context to the latent space (like word vector models had done in the past) but an entire sequence of words. Interestingly enough even with word vectors you can learn word vectors in language A and also language B and then manually align the latent spaces in order to get simple translation for close to free. So given the training data size of GPT-3+ 0.02% is a lot of language.