4 ms·
Explain me this. As I understand, chatgpt is trained by dumping a huge amounts of text in it. It doesn't have a notion of "language", all it does is continues t
by leshenka 3y ago
Explain me this. As I understand, chatgpt is trained by dumping a huge amounts of text in it. It doesn't have a notion of "language", all it does is continues the series of tokens. Therefore all you need to do in order to train it a new language is to build a corpus of text and feed it to the model. Is this right? And if so, is this what they did here or is it something completely different?
- Gravityloss 3y agoYes, Finnish works fine.
- tuukkah 3y agoTo begin with, the training material is tokenized and one thing the choice of the optimal tokenizer depends on is the source language(s). Consider that in English, the optimal tokenization of "cats" may be "cat" + "s" (based on the vocabulary and morphology of English) but in another language that may make no sense. The choice of the tokenizer is probably more important in a language which has less training material available.
- otabdeveloper4 3y agoMost likely it's just a shitty job of curating the training set. Estonian is related to Finnish and they probably got some of the texts mixed up, because who cares about all those silly languages, amirite?