Language Models trained at the size and data they tested are pretty terrible to begin with, translation or not. Like I said, the crucial thing is that it looks

Language Models trained at the size and data they tested are pretty terrible to begin with, translation or not. Like I said, the crucial thing is that it looks to be a diminishing gap. I'm not saying parallel corpora is doing nothing or is unimportant but that's a clue that monolingual competency of the languages in question is the biggest factor here. The extreme positive transfer language models exhibit in terms of multilingual competency is another clue. https://arxiv.org/abs/2108.13349 https://arxiv.org/abs/2108.13349 Parallel corpora may be a crutch with fading relevance.