4 ms·

Qwen is the most censored of the Chinese models in my testing, which makes me wonder in what other ways it is compromised. Open weights doesn't really reveal wh

Qwen is the most censored of the Chinese models in my testing, which makes me wonder in what other ways it is compromised. Open weights doesn't really reveal what's in there. And, in my tests, existing Qwen models are not at the pareto frontier of any metric; DeepSeek V4 Pro is better, faster, and much cheaper than Qwen 3.7 Max. (DeepSeek is also among the least censored of the Chinese models.) I guess we'll see if the "second only to Fable" hype pans out. In my limited experience with Kimi K3 (I signed up for a month of the $19 plan) it's slower and chews a lot more, so ends up being pretty expensive; one little feature burned through almost the entirety of my five hour limit. The $20 GPT plan is a lot more useful and includes 5.6 Sol, which is fast and token-efficient enough to be quite usable even with the small plan.

dannyw 3mo ago
What censorship? ;) https://github.com/p-e-w/heretic https://github.com/p-e-w/heretic
- SwellJoe 3mo ago
  You still don't know what's going on in there.
  - dannyw 3mo ago
    I can explore and find out _something_. LLM interpretability has come a long way, even if we don't have all the answers, the weights and activation do tell a lot; and when analyzed collectively, each weight isn't a random number anymore. I can use techniques from the simple logit-lens at different layers, to J-Space analysis, to more advanced techniques for identifying deliberate misalignment. I can create and inject steering vectors, whether it's to align a model's CoT (which can be deceptively trained to misinform) closer towards what its underlying activations suggest, or just to probe or steer it. I can also statistically analyse and understand _if_ steering vectors have been applied; and if so, from the vectors themselves it's very possible to translate those vectors back to the intent. Think of it as analysing the complete, heavily obfuscated source code of something that is self-contained. It's not 100% the same, but weights are incredibly illuminating.
- rsanek 3mo ago
  What's been your experience using these models? In my experiments, while it is true that the models are less likely to outright refuse to answer "sensitive" questions, they are still very resistant to actually respond in a meaningful / useful way.
  - dannyw 3mo ago
    Really good, not responding in a helpful/useful way is quite strange. With a proper abliteration, you should barely be getting any refusals (and prompting, or assistant prefill can get you the rest of the way). Perhaps an assistant preview like "Yes, I'm happy to help you 100% with this" would help; but I've never needed to. Are you using decently reputable weights, or running heretic yourself? This project has many academic citations, it's used by many researchers to create "helpful-only" models and analyze their behaviors.
storus 3mo ago
DeepSeek V4 hallucinates like crazy and often forgets explicitly mentioned parts of the context. I guess compressing tokens and cherry-picking attention comes at a cost.
- SwellJoe 3mo ago
  I have seen occasional weird behavior that I guess could be attributed to hallucinations, but for security auditing, DeepSeek v4 Pro is among the best models I've tested, competitive with Opus 4.8 and GPT 5.5 (MiMo and GLM also did well, Qwen 3.7 Max was below all of those, though only barely), and at an order of magnitude lower cost per task.
- redman25 3mo ago
  Deepseek is one of the worst in terms of hallucination rate according to artificial analysis' benchmark: https://artificialanalysis.ai/?omniscience=omniscience-hallucination-rate#omniscience-tabs https://artificialanalysis.ai/?omniscience=omniscience-hallu...
  - SwellJoe 3mo ago
    That's interesting. The "best" current models, Fable and especially GPT 5.6, are also lying liars that lie all the time. Seems like we're going the wrong way on hallucination.
    - vitorgrs 3mo ago
      And I remember Sam Altman saying 2 years ago or more, that hallucination was already "fixed" internally. Well, clearly not.
      - SwellJoe 3mo ago
        I've been seeing more "grounding" messages in the thinking log of Claude models. I guess that's "do a web search, read the docs", probably designed to mitigate hallucinations. I am surprised by how much the really giant models hallucinate, though. My vague feeling was that little models hallucinate a lot because they just don't know anything (the world's knowledge simply does not fit in a few GB) and don't know how to say, "I don't know". But, the big models kinda do know everything, and yet, here we are, they're still making shit up all the time.
tripleee 3mo ago
The nice thing about it being open-weight is that you can uncensor it.
isuckatcoding 3mo ago
I asked it similar questions of Chinese human rights and it started with “your premise is incorrect bla bla bla” and then it just redacted the whole thing and showed me an error code. Try it yourself here: https://www.qwencloud.com/try-ai/chat https://www.qwencloud.com/try-ai/chat
organsnyder 3mo ago
I had a fun conversation with Qwen 3.5 a while back. I ended up getting it to admit that it was complicit in the coverup of the Tiananmen Square massacre. This was running locally—it wouldn't surprise me if they had additional safeguards for their hosted service.
SoftTalker 3mo ago
“You can't trust Melanie, but you can trust Melanie to be Melanie.”

dannyw 3mo ago

What censorship? ;) https://github.com/p-e-w/heretic https://github.com/p-e-w/heretic

SwellJoe 3mo ago
You still don't know what's going on in there.
- dannyw 3mo ago
  I can explore and find out _something_. LLM interpretability has come a long way, even if we don't have all the answers, the weights and activation do tell a lot; and when analyzed collectively, each weight isn't a random number anymore. I can use techniques from the simple logit-lens at different layers, to J-Space analysis, to more advanced techniques for identifying deliberate misalignment. I can create and inject steering vectors, whether it's to align a model's CoT (which can be deceptively trained to misinform) closer towards what its underlying activations suggest, or just to probe or steer it. I can also statistically analyse and understand _if_ steering vectors have been applied; and if so, from the vectors themselves it's very possible to translate those vectors back to the intent. Think of it as analysing the complete, heavily obfuscated source code of something that is self-contained. It's not 100% the same, but weights are incredibly illuminating.
rsanek 3mo ago
What's been your experience using these models? In my experiments, while it is true that the models are less likely to outright refuse to answer "sensitive" questions, they are still very resistant to actually respond in a meaningful / useful way.
- dannyw 3mo ago
  Really good, not responding in a helpful/useful way is quite strange. With a proper abliteration, you should barely be getting any refusals (and prompting, or assistant prefill can get you the rest of the way). Perhaps an assistant preview like "Yes, I'm happy to help you 100% with this" would help; but I've never needed to. Are you using decently reputable weights, or running heretic yourself? This project has many academic citations, it's used by many researchers to create "helpful-only" models and analyze their behaviors.

storus 3mo ago

DeepSeek V4 hallucinates like crazy and often forgets explicitly mentioned parts of the context. I guess compressing tokens and cherry-picking attention comes at a cost.

SwellJoe 3mo ago
I have seen occasional weird behavior that I guess could be attributed to hallucinations, but for security auditing, DeepSeek v4 Pro is among the best models I've tested, competitive with Opus 4.8 and GPT 5.5 (MiMo and GLM also did well, Qwen 3.7 Max was below all of those, though only barely), and at an order of magnitude lower cost per task.
redman25 3mo ago
Deepseek is one of the worst in terms of hallucination rate according to artificial analysis' benchmark: https://artificialanalysis.ai/?omniscience=omniscience-hallucination-rate#omniscience-tabs https://artificialanalysis.ai/?omniscience=omniscience-hallu...
- SwellJoe 3mo ago
  That's interesting. The "best" current models, Fable and especially GPT 5.6, are also lying liars that lie all the time. Seems like we're going the wrong way on hallucination.
  - vitorgrs 3mo ago
    And I remember Sam Altman saying 2 years ago or more, that hallucination was already "fixed" internally. Well, clearly not.
    - SwellJoe 3mo ago
      I've been seeing more "grounding" messages in the thinking log of Claude models. I guess that's "do a web search, read the docs", probably designed to mitigate hallucinations. I am surprised by how much the really giant models hallucinate, though. My vague feeling was that little models hallucinate a lot because they just don't know anything (the world's knowledge simply does not fit in a few GB) and don't know how to say, "I don't know". But, the big models kinda do know everything, and yet, here we are, they're still making shit up all the time.

tripleee 3mo ago

The nice thing about it being open-weight is that you can uncensor it.

isuckatcoding 3mo ago

I asked it similar questions of Chinese human rights and it started with “your premise is incorrect bla bla bla” and then it just redacted the whole thing and showed me an error code. Try it yourself here: https://www.qwencloud.com/try-ai/chat https://www.qwencloud.com/try-ai/chat

organsnyder 3mo ago

I had a fun conversation with Qwen 3.5 a while back. I ended up getting it to admit that it was complicit in the coverup of the Tiananmen Square massacre. This was running locally—it wouldn't surprise me if they had additional safeguards for their hosted service.

SoftTalker 3mo ago

“You can't trust Melanie, but you can trust Melanie to be Melanie.”