3 ms·
It's still not necessary to have bilingual data https://news.ycombinator.com/item?id=38574712 https://news.ycombinator.com/item?id=38574712
by famouswaffles 3y ago
It's still not necessary to have bilingual data
https://news.ycombinator.com/item?id=38574712 https://news.ycombinator.com/item?id=38574712
- janalsncm 3y agoFirst of all, thank you for finding this paper, it seems to be addressing the question directly. However, I believe you are overstating the results of the paper. If anything, the paper demonstrated massive reductions in zero shot translation capabilities after removing translation. And for languages with no cognates with English, BLEU scores are pretty abysmal. So the paper suggests that parallel corpuses still very important. For whale languages, you won’t have any cognates with English, and we don’t even know if they have a grammar that is remotely human.
- famouswaffles 3y agoLanguage Models trained at the size and data they tested are pretty terrible to begin with, translation or not. Like I said, the crucial thing is that it looks to be a diminishing gap. I'm not saying parallel corpora is doing nothing or is unimportant but that's a clue that monolingual competency of the languages in question is the biggest factor here. The extreme positive transfer language models exhibit in terms of multilingual competency is another clue. https://arxiv.org/abs/2108.13349 https://arxiv.org/abs/2108.13349 Parallel corpora may be a crutch with fading relevance.