4 ms·
Masterpiece. I can only add that (quoting) the "rigid model of repeatedly appending single words to an immutable transcript" makes LLMs less suitable for gener
by patrakov 4y ago
Masterpiece.
I can only add that (quoting) the "rigid model of repeatedly appending single words to an immutable transcript" makes LLMs less suitable for generating text in certain spoken languages, because, by their very structure, they require a little bit of backtracking. In the case of the Russian language, the backtracking comes from the need to modify the ending of prior adjective to the grammatical gender and number of the noun that follows. I have not tested ChatGPT, but grammatical mistakes in this area are quite common with a Russian LLaMA derivative (see https://huggingface.co/IlyaGusev/llama_13b_ru_turbo_alpaca_lora_llamacpp/blob/main/README.md https://huggingface.co/IlyaGusev/llama_13b_ru_turbo_alpaca_l...).
- famouswaffles 4y agoI have access to GPT-4. What would you like to test ? I'm interested in seeing how it performs
- patrakov 4y agoI have tested the free version of ChatGPT, and so far in my limited testing it connects Russian words correctly. The problem (which cannot exist in English, because both one apple and multiple apples can be "green") is only apparent in less capable models.
- famouswaffles 4y agoThat's what i figured. The thing with language models is that there seems to clearly be some kind of behind the veil...thinking ? There are some completions that are impossible without some form of thinking ahead. The only issue is that this "thought"/state can't be transferred to another generation without being written down.