2 ms·
How does this work? It seems like it, in most ways, it would be bad to train on 24 separate languages. That's just 24 partitions to the data. Seems really inef
by nonethewiser 1y ago
How does this work?
It seems like it, in most ways, it would be bad to train on 24 separate languages. That's just 24 partitions to the data. Seems really inefficient and better to simply train in the biggest (english) and translate.
I do think this will introduce some biases that correlate with the English language. It would be interesting to see more specifically what this means. But regardless, I don't think you can produce a competitive model with such a large subdivision of training data.
- antiloper 1y agoIf you train a model on multiple languages, you can use the model itself for translation. As well as allowing the model to naturally respond in the user's language.
- whimsicalism 1y agonah, it's better to train on all languages. 24 partitions? you are gravely underestimating these models and how they represent things in their latents... transfers easily