4 ms·
I noticed similar behaviour in an older model (Skywork 13B) a few months back. When asked in Chinese, it would politely say that nothing of note occurred when r
by mattstir 3y ago
I noticed similar behaviour in an older model (Skywork 13B) a few months back. When asked in Chinese, it would politely say that nothing of note occurred when responding to queries about Tiananmen Square, etc. In English, it would usually respond truthfully. It was deliberate in the case of Skywork, based on their model card (https://huggingface.co/Skywork/Skywork-13B-base https://huggingface.co/Skywork/Skywork-13B-base):
> We have developed a data cleaning pipeline with great care to effectively clean and filter low-quality data and eliminate harmful information from text data.
I'd imagine it's likely similar for Yi.
- BoorishBears 3y agoHuge jump to go from that line in the model card to it being intentional from the model's creators. China censors those events. They pre-trained with a specific focus on Chinese text, and integrated more native Chinese text than most models do. Doesn't require any additional filtering on their behalf to have the model reflect that, and if anything the fact that they're mentioned in english implies the opposite of your hypothesis. If they were going to filter Tiananmen Square, the lift to filter it in English would not be any higher.