3 ms·
For example: Slovene language. You simply don't have enough data on it. But if you add all the data that is available on related languages, you will get a highe
by miros_love 1y ago
For example: Slovene language. You simply don't have enough data on it. But if you add all the data that is available on related languages, you will get a higher quality. LLM fails with this property for low-resource languages.
- yorwba 1y agoThey train on 14 billion tokens in Slovene. Are you sure that's not enough?
- miros_love 1y agoUnfortunately, yes. We need more tokens, more variety of topics in texts and more complexity.
- mdp2021 1y agoWe need one-shot learning. (That amount is equivalent to 50000 books, which few nationals will have read.)
- Etheryte 1y agoI'm not sure I'm convinced. I speak a small European language and the general experience is that LLMs are often wrong exactly because they think they can just borrow from a related language. The result is even worse and often makes no sense whatsoever. In other words, as far as translations go, confidently incorrect is not useful.