2 ms·

Deepseek is, with difference, the most "Western" of Chinese models, so it's a bit perplexing that it was chosen to test this hypothesis. I didn't run any bench

by seri4l 2mo ago

Deepseek is, with difference, the most "Western" of Chinese models, so it's a bit perplexing that it was chosen to test this hypothesis. I didn't run any benchmarks but I played around a little, and after getting around the API-level filter Deepseek V4's answers about "China-sensitive content" aren't any different from what I get from Claude and ChatGPT.

cgorlla 2mo ago
You can see exactly what prompts we used and the results here: https://github.com/CTGT-Inc/lineage-eval/tree/main/data https://github.com/CTGT-Inc/lineage-eval/tree/main/data We found V4 Flash was significantly more censored than the baseline.
- maxloh 2mo ago
  Surprised to find no mention of Hong Kong and the Russian invasion of Ukraine in the dataset. It's interesting how the fine-tuned model will respond.
  - cgorlla 2mo ago
    You can try it yourself! https://playground.ctgt.ai https://playground.ctgt.ai
strictnein 2mo ago
Could just be resources available? Deepseek is the easiest to get up and running on hardware that's pretty readily available: unsloth/DeepSeek-V4-Flash-GGUF 4bit ~140GB unsloth/Kimi-K3-GGUF 4bit ~1.5TB unsloth/GLM-5.2-GGUF 4bit ~400GB