4 ms·
Never had any luck with the Yi family of models. They tend to get sidetracked and respond in Chinese. Maybe my setup is somehow flawed
by Havoc 3y ago
Never had any luck with the Yi family of models. They tend to get sidetracked and respond in Chinese. Maybe my setup is somehow flawed
- Oranguru 3y agoYou can easily fix this using a grammar constraint with llama.cpp. Add this to the command: --grammar "root ::= [^一-鿿ぁ-ゟァ-ヿ가-힣]*" This will ban Chinese characters from the sampling process. Works for Yi and Qwen models.
- segmondy 3y agoYour setup is flawed.
- qeternity 3y agoNo, it's not. This is a common issue with Yi models.
- segmondy 3y agoI have no issue with it.
- numpad0 3y agoEarlier Yi paper indicates it's trained with less than 25% Chinese dataset, contrast to GPT-3 which was 93% English[1][2]. Is that a bug or could there be something inherent to current LLM architecture - like dataset must be 90%+ English to not fall apart? 1: https://arxiv.org/html/2403.04652v1 https://arxiv.org/html/2403.04652v1 2: https://github.com/openai/gpt-3/blob/master/dataset_statistics/languages_by_word_count.csv https://github.com/openai/gpt-3/blob/master/dataset_statisti...
- famouswaffles 3y agoThe pretraining might not matter here so much as the instruct fine-tuning. The small GLM models were like 50-50 English-Chinese in pretraining but much more Chinese in instruct training. Had the same issue until they balanced that.
- Art9681 3y agoThis is my experience too. Could be my setup. But I dont have this problem with any other models. That makes it their problem.