They don't perform poorly (depends on how much corpus training the language received). https://arxiv.org/abs/2108.13349 https://arxiv.org/abs/2108.13349. The im

They don't perform poorly (depends on how much corpus training the language received). https://arxiv.org/abs/2108.13349 https://arxiv.org/abs/2108.13349. The important thing is you don't need an equivalent number of tokens for an equivalent performance. So even though GPT-3 only had 1.8 % french by word count, it can speak fluent french. There's positive transfer between languages. Also GLM-130B trained on equal English/Chinese (200b tokens each. 400b total) speaks both just fine.